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FOREWORD 


The  Hunan  Factors  Technical  Area  of  the  Army  Research  Institute  (ARI) 

Is  concerned  with  human  resource  demands  of  increasingly  complex  battlefield 
systems  used  to  acquire,  transmit,  process,  disseminate  and  utilize  informa¬ 
tion.  This  increased  complexity  places  great  demands  on  the  operator  inter¬ 
acting  with  the  system.  Research  in  this  area  focuses  on  human  performance 
problems  related  to  interactions  within  coonand  and  control  centers  as  well 
as  issues  of  system  development.  The  research  program  includes  both  technology 
base  and  advanced  development  research  as  well  as  a  limited  amount  of  technical 
advisory  service  (TAS)  to  Army  agencies  and  activities. 

One  area  of  special  interest  involves  the  development  of  estimates  for  the 
contributions  of  human  factors  in  military  system  development.  The  inquiry 
into  this  topic  resulted  from  a  tri-service  committee  decision  to  investigate 
the  possibility  of  providing  system  designers/managers  with  evidence  of  the 
value  of  human  factors  to  compare  with  other  pertinent  information  from 
engineers,  operations  research  analysts  and  system  analysts.  This  final 
report  applies  the  impact  assessment  methodology  of  a  previous  report 
(TR  476)  to  two  military  systems  to  demonstrate  the  methodology's  feasibility. 
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MEASURING  AND  ENHANCING  THE  CONTRIBUTION  OF  HUMAN  FACTORS  IN 
MILITARY  SYSTEM  DEVELOPMENT:  CASE  STUDIES  OF  THE  APPLICATION 

of  impact  Assessment  methodologies 

BRIEF 


Requirement: 

To  demonstrate  the  applicability  of  the  recommended  con¬ 
ceptual  methodology  for  assessing  the  contribution  of  human 
factors  in  military  system  development.  Also,  to  outline  the 
contents  of  a  human  factors  impact  assessment  handbook. 

Procedure : 

Fulfilling  the  requirement  was  a  three-step  process: 

#  First,  impact  analysis  was  selected  as  the  appropriate 
methodology;  two  case  studies,  a  generic  maneuver 
control  system  and  the  F/A-18,  were  then  selected  for 
demonstrating  the  methodology.  The  steps  in  the 
methodology  were  presented,  and  questions  related  to 
metrics,  impact  areas,  and  available  case-system  docu¬ 
mentation  were  addressed.  A  plan  for  implementing  the 
methodology  emerged  from  this  effort. 

e  The  plan  for  demonstrating  the  feasibility  of  impact 
analysis  was  carried  out.  From  available  documentation, 
human  factors  products  were  assembled  for  the  case 
systems,  and  information  about  tradeoffs,  system 
requirements,  costs,  decisions,  constraints,  and  design 
deficiencies  were  derived.  The  human  factors  issue 
selected  for  the  F/A-18  was  foot  clearance  during 
ejection;  in  the  maneuver  control  case,  the  issue 
selected  was  the  question  of  the  dedicated  versus 


non-dedicated  (console)  user /operator.  Impact  analysis 
was  conducted  to  assess  the  cost- benefits  of  different 
design  options  in  the  two  cases. 

•  The  rationale^  information/  and  conclusions  of  the 

project  to  this  point  served  as  the  basis  for  a  handbook 
outline.  The  handbook  will  be  intended  for  the  systems 
development  community  and  will  illustrate  the  importance 
of  both  utilizing  and  evaluating  human  factors  during 
systems  acquisition. 

Product : 

The  results  of  the  case  studies  indicate  that  impact 
analysis  is  a  feasible  method  of  assessing  the  cost-benefits 
of  human  factors  in  systems  development.  Conclusions  were  also 
reached  in  regard  to  the  role  of  the  human  factors  practitioner, 
the  documentation  of  human  factors  products,  and  the  adminis¬ 
trative  aspects  of  human  factors  in  the  systems  context. 

Utilization: 

The  demonstration  of  the  methodology  illustrates  that  there 
is  a  tool  available  both  for  presenting  the  contributions  of 
human  factors  in  a  tangible  form  and  for  helping  to  make  allo- 
cational  decisions  and  tradeoffs  which  involve  human  performance 
and  compatibility  issues.  The  conceptual  basis  and  applicability 
of  the  methodology  will  contribute  to  the  handbook  which  is  to 
be  developed.  Also,  issues  raised,  and  insights  achieved,  during 
the  course  of  the  project  will  contribute  to  that  undertaking . 
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CHAPTER  1 
INTRODUCTION 

This  report  represents  the  second  phase  of  a  multiphase 
project  intended  to  demonstrate  both  clearly  and  precisely  the 
contributions  of  human  factors  to  the  development  of  military 
systems.  The  problem  is  an  evolutionary  one  in  the  sense  that 
the  growing  sophistication  of  our  technology,  along  with  policy- 
level  incentives  for  the  immediate  application  of  that  sophisticated 
technology  to  military  systems,  has  outstripped  our  capabilities 
for  ensuring  the  incorporation  of  human  factors  in  the  design  of 
these  systems.  This  gap  has  occurred  in  spite  of  a  re-growth  of 
recognition  of  the  problem  on  the  part  of  weapons  system  develop¬ 
ment  specialists  both  within  and  outside  the  official  establishment. 
Although  Department  of  Defense  (DOD)  Directives  5000.1,  5000.2, 
and  5000.3  specify  a  requirement  for  human  factors  input  during 
each  of  the  successive  phases  of  the  system  acquisition  cycle, 
human  factors  (HF)  considerations  often  are  delayed  until  after 
the  basic  configuration  of  the  system  has  been  fixed.  At  that 
point,  human  factors  can  address  only  the  subsidiary  questions 
of  task  sequences,  manning  level,  training  requirements, 
interfaces,  etc. ;  the  HF  staff  is  not  able  at  that  stage  to 
contribute  to  the  assessment  of  broader  conceptual  issues,  which 
should  be  addressed  at  the  very  beginning  of  the  development 
process.  Consequently,  the  staff  is  often  confronted  with  human 
factors  problems  that  might  not  have  existed  in  the  first  place 
had  their  expertise  been  employed  at  the  initial  planning  stage. 

A  contributing  problem  has  been  the  lack  of  a  methodology 
for  evaluating  objectively  the  contributions  of  human  factors  at 
any  stage  of  system  development.  This  problem  of  assessing 
human  factors  contributions  is  central  to  the  direction  and 
final  objectives  of  the  present  effort. 


Initial  Project  Goals 


The  allocation  of  resources  for  any  particular  endeavor 
within  the  systems  development  context  rests  upon  the  premise 
that  such  an  endeavor  has  previously  made  a  measurable  contribution 
to  the  development  of  a  system.  In  this  regard,  the  importance 
of  objective,  quantifiable  data  for  decision-making  in  the 
development  process  is  apparent  to  anyone  involved.  The  use  of 
formal  mathematical  models  or  their  near  equivalent  has  permeated 
every  level  of  system  development  from  the  engineering  draftsman 
to  the  top  level  of  the  DOD.  The  basic  philosophy  is  that  every 
decision  carries  with  it  quantifiable  costs  and  benefits  which 
must  be  weighed  against  those  of  alternative  decisions.  There 
are  many  different  cost/benefit  models,  but  they  all  employ  the 
same  fundamental  logical  structure:  Given  specific  system 
goals,  alternative  solutions  to  system- related  problems  must  be 
thoroughly  evaluated  for  their  relative  and/or  absolute  impacts 
upon  the  objectives  required  to  meet  those  goals.  Depending 
upon  the  nature  of  both  the  system  and  the  problem  to  be 
resolved,  such  impacts  can  be  quantified  with  varying  degrees  of 
ease  or  difficulty;  the  assumptions  made  and  the  methodology 
used  depend  to  a  large  extent  upon  the  difficulties  encountered 
in  deriving  numerical  impact  data.  Of  several  alternative 
solutions,  the  preferred  one  is,  of  course,  the  one  that  is 
predicted  to  roost  closely  meet  the  criteria  established  for  a 
successful  system  at  the  lowest  cost.  The  reliability  of 
quantitative  impact  data  is  therefore  a  crucial  factor  in  the 
decision-making  process. 

Cost/benefit  analyses  have  been  employed  in  various  systems- 
related  areas,  such  as  logistics,  ordnance  engineering,  and 
equipment  reliability.  These  areas  all  permit  the  derivation  of 
"hard"  data  which  can  be  transformed  into  probabilistic  inpacts, 
and  for  this  reason  such  applications  of  cost/benefit  analyses 


have  a  high  degree  of  acceptability.  The  primary  purpose  of  the 


application  of  cost/benefit  analysis  to  human  factors  outputs. 

Case  studies  were  constructed  around  selected  episodes  in 
specific  military  systems  programs.  The  implementation  of  the 
case  studies  provides  the  groundwork  for  handbooks  intended  to 
sensitize  system  designers  to  human  factors  issues,  and  human 
factors  specialists  to  the  needs  of  system  designers. 

Progress  of  the  Ongoing  Project 

Phase  I 

Phase  I  of  this  study  (see  Price  et  al.,  1980)  led  to 
a  determination  of  (1)  a  conceptual  basis  for  human  factors 
contributions  to  military  systems  development,  and  (2)  a  feasible 
method  for  evaluating  the  contribution  of  human  factors  (see 
Exhibit  1-1) .  This  phase  explored  the  HF  process,  examples  of 
HF  contributions  and  problems,  the  systems  acquisition  cycle, 
and  cost  analytic  methods  having  potential  applicability  for 
assessing  HF  contributions  to  military  systems.  Out  of  this 
phase  emerged  two  concepts  of  particular  importance: 

•  The  Principal  Human  Factors  Product 

•  Impact  Analysis. 

The  principal  product  is  the  cumulative  outcome  of  the  human 
factors  process.  Each  stage  of  the  development  cycle  yields  its 
own,  unique  product.  Since  the  effects  of  HF  actions  are 
cumulative,  the  principal  product  of  one  stage  is  only  as  valuable 
as  those  that  were  achieved  in  the  preceding  stages.  In  general, 
it  can  be  assumed  that  the  principal  product  ia  the  contribution  of 
human  factors  to  the  system. 


Exhibit  1-1 

RELATIONSHIP  OF  THE  MAJOR  EFFORTS  OF  THE  STUDY 


the  Value  of  Human  Ft 
in  Military  System  Davet 


Ami  assessment  of  HF  decisions/allocations  integral  to  a 
product*  as  against  alternative  decisions/allocations*  leads  to 
the  second  concept,  impact  analysis.  Impact  analysis  is  the 
methodology  developed  as  the  analytical  tool  for  measuring  the 
cost/benefits  of  human  factors  actions,  assessing  such  actions 
in  terms  of  three  impact  areas:  cost,  capability,  and  compatibility. 
The  concept  of  impact  analysis,  along  with  the  concept  of  the 
principal  product,  provided  the  thrust  for  the  next  phase  of  the 
study. 

Phase  II 

This  phase  consisted  of  a  demonstration  of  the  cost/benefit 
methodology  and  was  constructed  around  the  principal  product 
concept  and  impact  analysis.  This  was  a  two-step  process 
consisting  of  (1)  an  elaboration  of  the  impact  assessment 
methodology,  and  (2)  a  demonstration  of  the  methodology  using 
two  case  study  systems.  The  case  studies  involved,  first,  the 
derivation  of  tentative  principal  products  for  a  selected 
development  stage  of  each  of  the  two  case  systems,  and  then  the 
application  of  the  methodology  to  the  assessment  of  alternative 
human  factors  actions.  The  case  studies  are  presented  in 
Appendix  A. 

Changing  Perspectives  and  Revised  Goals 

Much  of  the  major  effort  in  the  conduct  of  Phase  II  was 
devoted  to  the  examination  of  documents  for  the  purpose  of 
extracting  human  factors  infozmation  related  to  the  two  case 
systems.  The  difficulties  encountered  and  findings  obtained 
from  this  task  led  to  important  conclusions  which  bear  upon  the 
goals  of  the  project;  they  are  as  follows: 


1-5 


•  Few  if  any  real  systems  are  being  developed  in  accordance 
with  the  formal  sequence  prescribed  by  the  Secretary  of 
Defense. 

e  The  critical  events  in  the  systems  development  process 
are  not  the  formalized  milestones,  but  rather  the 
formation  of  the  MENS  team,  the  appointment  of  the  PM, 
and  the  selection  of  a  prime  contractor. 

e  The  key  events  often  do  not  reflect  the  participation 
of  human  factors  specialists. 

e  A  clear  picture  of  the  human  factors  principal  product 
does  not  emerge  from  systems  documentation.  Thus,  the 
accountability  for,  and  contributions  from,  key  HF 
decisions  are  elusive. 

These  conclusions  have  implications  for  both  the  concepts 
of  the  ongoing  project  and  the  target  audiences  of  the  proposed 
handbook.  In  respect  to  the  first,  the  principal  product  should 
be  thought  of  not  only  as  the  net  HF  contribution  for  any  stage 
of  systems  development  but  also  as  the  documentation  process 
necessary  for  exhibiting  such  a  contribution.  Also,  the  principal 
products  must  be  sufficiently  flexible  in  content  that  they  can 
be  adapted  to  the  real-world  deviations  from  the  idealized 
acquisition  cycle.  With  respect  to  the  target  audience  for  the 
handbook,  three  parties  in  particular  should  receive  attention: 
the  combat  developer,  the  HF  staff,  and  the  PM  and  his  staff. 

The  combat  developer  needs  to  be  acquainted  with  the  crucial  HF 
issues  which  should  be  addressed  during  the  Mission  Analysis 
Phase.  The  HF  staff  should  be  aware  of  the  importance  of 
documenting  the  HF  process  in  order  to  establish  credibility; 
the  staff  must  also  be  made  aware  of  the  need  to  assert  itself 
to  support  the  PM  and  lend  assistance  to  the  combat  developer. 
Finally,  the  PM  needs  to  be  aware  of  both  the  importance  of  HF 
considerations  and  the  resources  which  the  HF  staff  has  at  its 
disposal. 
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Chapter  2  reviews  the  conceptual  basis  of  HF  contributions 
in  more  detail,  focusing  on  the  following:  examples  of  systems 
HF  deficiencies  and  utilization;  the  military  systems  procure¬ 
ment  cycle;  and  the  principal  product  concept.  Chapter  3  reviews 
the  cost/benefit  approach  employed.  Chapter  4  presents  conclusions 
and  recommendations  regarding  the  utilization  of  the  cost/benefit 
methodology  and  the  role  of  the  human  factors  practitioner.  In 
general,  care  has  been  taken  to  reflect  conceptual  changes 
occurring  during  the  course  of  the  project.  Finally,  Appendices 
A  and  B  contain,  respectively,  the  case  studies  and  an  outline 
of  the  handbooks  proposed  for  the  next  project  phase. 


CHAPTER  2 

REVIEW  OF  THE  CONCEPTUAL  BASIS 
FOR  HUMAN  FACTORS  CONTRIBUTIONS 

The  first  step  in  assessing  the  contributions  of  human 
factors  to  military  systems  was  a  discussion  of  the  rationale  for 
human  factors  utilization;  some  examples  of  system  failures  due 
to  inadequate  HF  were  central  to  this  discussion.  This  review 
was  followed  by  a  formal  chronology  of  the  weapon  system  procure¬ 
ment  cycle  and  related  human  factors  efforts,  a  discussion  of  the 
principal  product  concept,  and  the  conceptualization  of  the 
impact  analysis  framework.  The  present  chapter  presents  a  digest 
of  these  areas  with  the  exception  of  the  impact  analysis  framework, 
which  is  Chapter  3.  The  final  chapter  (Chapter  4)  makes  recommen¬ 
dations  based  upon  conclusions  reached  during  the  conduct  of  this 
project. 

HF  Problems  and  Applications 

With  few  possible  exceptions,  it  would  be  unfair  to  label 
most  systems  as  "failures"  or  "successes"  entirely  on  the  basis 
of  the  degree  to  which  HF  has  been  applied  during  systems  develop¬ 
ment.  However,  it  is  fair  to  say  that  the  value  of  HF  to  the 
success  of  a  system  is  disproportionately  greater  than  the  resources 
usually  allotted  to  this  activity.  Some  systems  operate  poorly 
because  of  a  lack  of  HF  considerations;  others  appear  to  be 
successful  in  large  part  because  they  were  designed  with  the 
human  operator  or  maintainer  in  mind.  Thus,  it  will  be  instruc¬ 
tive  to  briefly  discuss  some  specific  systems  and  design  decisions 
in  which  either  attention  or  inattention  to  HF  has  had  a  substantial 
bearing  upon  performance. 


Examples  of  HF  Neglect  and  Success 


Although  many  modern-day  systems  are  both  huge  and  extremely 
complex,  difficulties  related  to  poor  human  factors  are  encountered 
in  even  those  systems  which  appear  to  be  relatively  simple.  A 
recent  article  by  Fallows  (1981a)  points  out  that  the  Dragon,  a 
hand-held  missile- launcher  employed  against  tanks  at  short  range, 
has  limited  utility  because  its  operation  conflicts  markedly  with 
basic  battle-field  requirements.  The  operator  must  guide  the 
missile  toward  the  target  over  a  period  of  seconds  while  standing 
exposed  on  the  battlefield.  Not  only  is  he  himself  a  target 
during  this  period,  but  he  has  the  additional  problem  of  maintaining 
a  heavy  weapon  on  his  shoulder  without  moving  the  sight  following 
a  heavy  blast.  This  requires  a  rare  combination  of  strength  and 
skill.  Similar  problems  exist  with  the  TOW,  a  long-range  missile 
launcher  also  operated  by  a  single  person.  Fallows  (1981b)  has 
discussed  design  problems  in  more  conventional  small  arms  as 
well.  For  example,  the  Ml 6  was  fitted  with  an  additional  bolt 
handle  which  increased  the  tendency  of  soldiers  to  attempt  the 
seating  of  jammed  cartridges,  behavior  that  leads  to  more  severe 
jamming  and  can  damage  the  weapon. 

Given  that  design  features  of  small  systems  can  adversely 
affect  their  operation  by  increasing  the  probability  of  human 
error,  the  potential  influence  of  inadequate  HF  upon  the  per¬ 
formance  of  operators  in  increasingly  large  and  complex  systems 
is  understandably  great.  A  dramatic  case  in  point  is  presented 
in  Exhibits  2-1  and  2-2.  These  exhibits  are  from  recent  briefings 
on  the  human  factors  program  at  the  Naval  Air  Development  Center. 
Exhibit  2-1  indicates  the  dual  problem  of  increasing  information 
requirements  for  aircraft  operators  and  the  decreasing  amount  of 
cockpit  space  available  for  providing  displays  or  controls.  As 
may  be  seen,  the  last  (and  newest)  weapon  system  on  that  chart, 
the  AV-8  (Harrier)  V/STOL  aircraft,  has  approximately  one-third 
the  cockpit  space  that  the  F-4  aircraft  has. 


2-2 


Exhibit  2-1 


Cockpit  Space  and  Information  Requirement* 
for  Several  Navy  Aircraft  Weapon  System* 
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Exhibit  2-2 

Trend  of  Accident  Rates  for  Typical  Navy  Aircraft  and  the  AV-8  Harrier 
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Exhibit  2-2  shows  that  the  V/STOL  accident  rate  has 
been  increasing  the  la&t  few  years.  This  is  contrary  to  the 
experience  typically  encountered  when  new  aircraft  are 
introduced.  Furthermore,  "pilot  factor"  as  a  contributing 
cause  seems  to  be  high.  With  respect  to  this  last  point, 
the  data  shown  in  Exhibit  2-2  represent  21  accidents,  16  of 
which  occurred  in  the  V/STOL  flight  regime  (i.e.,  conversion 
flight,  landing,  or  takeoff).  Of  these  16,  11  had  pilot 
factor  as  a  contributing  cause.  It  should  also  be  added 
that  the  Naval  Air  Development  Center  has  since  initiated  a 
program  to  provide  human  factors  support  early  in  the  design 
of  V/STOL  aircraft. 

Another  prime  example  of  systems  complexity  impinging 
upon  accurate  operator  performance  is  the  automated  C^i 
system.  Commanders  have  to  make  crucial  decisions  on  the 
basis  of  information  flowing  in  from  often  remote  locations. 
Since  the  information  is  filtered,  distributed,  and  analyzed 
at  numerous  points  prior  to  its  arrival  at  the  Command  Post, 
the  allocation  of  these  functions  to  man  and  machines  is 
critical.  The  Tactical  Operations  System  (TOS)  experience 
exemplifies  what  can  happen  if  substantive  HF  efforts  are 
not  made  early  in  the  system  development  process.  A  combi¬ 
nation  of  the  message  flow  design  and  the  hardware/ software 
configuration  created  a  situation  in  which  most  of  the 
system  nodes  became  information  bottlenecks.  In  fact,  the 
amount  of  information  flowing  into  any  one  point  was  so  great 
that  it  is  questionable  whether  even  a  concentrated  HF 
effort  could  have  achieved  an  operable  system  once  the 
configuration  was  fixed. 

Many  human  factors  difficulties  would  appear  at  first 
glance  to  be  "common  sense"  matters,  and  easily  corrected. 

A  recent  Navy  publication  (Office  of  the  Assistant  Secretary, 
1980)  reports  the  following  HFE  deficiencies  on  naval  crafts 


•  Two  systems  having  similar  gas  turbine-powered  propulsion 
plants  have  entirely  different  display  boards  and  control 
systems  for  these  units. 

•  Different  passive  sonar  systems  use  different  processes 
and  display  formats  for  presenting  similar  types  of 
information. 

•  Filters  on  the  SPS-40B  Surface  Radar  cannot  be  reached 
for  routine  maintenance. 

•  The  emergency  cut-off  for  the  HP  Air  Flasks  is  difficult 
to  locate  and  reach  on  the  DD-963. 

•  The  access  hatch  to  the  Turbine  on  the  DD-963  is  blocked 
by  a  catwalk. 

Such  problems  are  not  so  common- sensical  or  easily  solved  as  one 
might  guess.  In  many  cases  the  total  system  configuration  was 
not  sufficiently  conceived  of  in  light  of  operator  and  maintainer 
needs,  which  was  also  the  essential  problem  with  the  Army's  TOS. 
Consequently,  an  HF  retrofit  might  mean  reconfiguring  a  bulkhead 
or  hatch,  or  retraining  operators  and  maintainers. 

The  preceding  discussion  describes  what  can  happen  if  HF  is 
neglected.  The  reverse  side  of  the  coin  is:  what  is  accomplished 
if  HF  expertise  is  heeded  at  critical  points  in  the  design 
process?  Accomplishments  cannot  be  pinpointed  as  dramatically  as 
failures,  since  a  smooth-working  system  does  not  attract  the  same 
attention  as  one  that  fails  in  its  mission  and/or  is  associated 
with  error-related  casualties  and  breakdowns.  However,  the 
article  cited  above  makes  a  good  case  for  the  successful  appli¬ 
cation  of  human  factors  in  naval  ships  and  aircraft,  as  reflected 
in  the  following  excerpt: 


There  is  ample  evidence  that  the  problems  observed  can 
be  solved.  Many  cases  of  excellent  man-machine  interface 
design  were  noted.  The  engine  and  electrical  control 
consoles  on  the  DD-985,  for  example,  were  well  laid  out 
and  easy  to  read.  The  SLQ-32  and  the  WR-12  are  examples 
of  easy-to-use  systems.  Outside  of  the  surface  community, 
much  can  be  learned  from  positive  examples  available  in 
aircraft  design.  For  example,  the  A-6  aircraft  reflects 
careful  attention  to  operator  interfaces,  with  the  result 
that  the  crew  can  effectively  perform  extraordinarily 
demanding  and  hazardous  missions.  Similar  attention  is 
being  given  to  man-machine  interfaces  in  the  F-18. 

One  need  not  be  restricted  to  the  Navy  to  uncover  HF  appli¬ 
cations.  Both  the  Air  Force  and  the  Army  have  devoted  substantial 
effort  to  optimizing  the  man-machine  design  characteristics  of 
system  under  development.  In  systems  as  different  as  SOTAS  and 
TITAN,  changes  in  the  spatial  and  functional  task  arrangements  of 
personnel  have  succeeded  in  improving  performance.  Human  factors 
applications  in  the  Air  Force  are  concentrated  in  the  aerospace 
area,  while  in  the  Army  such  efforts  are  directed  at  an  increasingly 
complex  array  of  systems:  small  arms,  anti-tank  weapons,  armored 
vehicles,  helicopters,  command-control,  etc.  Such  issues  as  man- 
hine  compatibility,  portability,  information  overload,  and 
computer  interface  are  of  special  importance.  Various  specific 
applications  could  be  enumerated,  but  it  should  suffice  to  point 
out  that  the  work  of  HF  staffs  has  contributed  to  the  design  of 
many  systems,  such  work  consisting  of  design  evaluations/ 
recommendations,  field  tests,  user  requirements  analyses,  task 
analyses,  training  evaluations,  etc.  A  series  of  reports  by  the 
Navy  (Price  &  Sands,  1978;  Price  &  Sands,  1979;  Lewis  et  al.,  1980; 
Lewis  et  al.,  1981)  and  the  Army  (Army  Research  Institute,  1980) 
provides  a  comprehensive  overview  of  such  efforts. 

Summary 

The  need  for  strong  HF  considerations  in  systems  development 
should  be  clear  by  this  point.  Other  examples  are  provided  in 
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in  the  Phase  I  report  (Price  et  al . ,  1980).  The  viability  of  a 
system  will  be  affected  by  its  compatibility  with  human  performance 
regardless  of  the  size,  complexity,  or  purpose  of  that  system. 

To  some  degree  poorly  human  engineered  systems  can  be  improved 
with  HF  fixes,  but  too  often  these  are  only  palliative  measures, 
due  to  the  increasing  inflexibility  which  accrues  in  the  design 
of  a  system  as  it  progresses  into  the  development  cycle.  Too 
often,  system  developers  omit  a  thorough  human  factors  "front-end 
analysis,"  and  the  performance-related  design  factors  create 
inadvertent  problems  that  are  compounded  as  the  development 
process  goes  forward. 

The  Chronology  of  Weapon  System  Procurement 

Basic  Model 

The  Phase  I  report  included  a  description  of  the  system 
acquisition  model  that  has  been  established  for  major  military 
systems.  The  model  is  based  on  the  guidelines  and  policies  for 
major  government  acquisitions  outlined  in  OMB  Circular  No.  A-109 
(1976) .  The  purpose  of  the  circular  is  to  foster  the  integration 
of  numerous  factors  (e.g.,  system  requirements,  costs,  land 
concepts)  in  order  to  avoid  past  problems  of  cost  overruns  and 
premature  commitments  to  full-scale  development  and  production. 

DOD  Directives  5000.1,  5000.2,  and  5000.3  respectively  provide 
policy,  policy /procedures,  and  test/e valuation  guidance  for  the 
acquisition  process  as  it  applies  to  military  systems.  These 
directives  recently  were  revised  to  effectively  augment  their 
relationship  to  requirements  for  human  factors  R&D,  the  func¬ 
tional  and  detailed  requirements  for  which  are  contained  in 
military  standards  MIL-STD-1472B  and  MIL-H-46855.  The  basic 
developmental  process  is  illustrated  in  Exhibit  2-3.  There  are 
four  essential  phases  which  precede  production  and  deployment, 
and  each  phase  may  be  conceived  of  as  follows: 


ACTION 


EVALUATION 
(MENS  /  SARC) 


DECISION 

(MILESTONE) 


The  phases  leading  up  to  the  different  milestones  (MI)  are  the 
following: 

Mission  Analysis  (precedes  MI  0)— -A  comparison  between  the 
present  technology  status  and  what  is  needed. 

Concept  Development  (precedes  MI  1) — A  study  of  the  strengths 
and  weaknesses  of  proposed  alternative  systems. 

Demonstration  and  Validation  (precedes  MI  2) — a  competitive 
demonstration  of  the  chosen  system (s) . 

Full-Scale  Development  (precedes  MI  3) — The  building  and 
testing  of  the  complete  syste&n. 

The  evaluation  mechanisms  for  each  phase  except  Mission  Analysis 
are  the  (S)SARCs  and  DSARCs,  the  service  and  DOD-level  formal 
reviews.  The  evaluation  of  the  MENS  by  the  Secretary  of  Defense 
constitutes  the  MI  0  evaluation.  The  purpose  of  the  evaluations 
is  to  determine  the  viability  of  the  system  concept  and  progress. 
Based  upon  the  conclusions  and  recommendations  emerging  from 
these  evaluations,  a  decision  is  reached  concerning  whether  or 
not  to  continue  the  development  effort  and  concerning  what 
modifications  might  be  incorporated,  given  continuation. 
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Decision  Points  for  Human  Factors 


The  trend  in  DOD  and  Service-level  documentation  is  to 
require  human  factors  participation  in  all  of  the  MI  phases  of 
major  systems  development.  But  one  of  the  conclusions  reached 
during  the  conduct  of  the  present  project  is  that  human  factors 
receives  the  least  consideration  when  it  is  needed  the  most, 
during  and  immediately  after  the  Mission  Analysis  phase.  Exhibit  2-4 
depicts  decision  points  which  are  critical  to  the  basic  design 
of  a  system  and  to  the  exercise  of  human  factors  throughout  the 
development  cycle.  The  development  of  the  MENS  statement  rarely 
includes  direct  input  from  HF  laboratory  or  staff  personnel. 

Since  the  system  requirements  follow  directly  from  the  MENS,  the 
development  of  the  system  in  respect  to  human  requirements  often 
can  be  incomplete.  The  appointment  of  the  PM  is  also  critical 
in  that  his  selection  and  monitoring  of  the  prime  contractor 
should  reflect  a  reasonably  keen  appreciation  of  human  factors. 

The  present  report  addresses  these  problems,  specifically  in  the 
redefinition  of  the  HF  practitioners'  role  and  in  the  proposed 
handbook  for  PMs. 


Exhibit  24 

Crucial  Points  for  Early  HFE  Inputs 


Variations  from  the  Framework  of  the  Acquisition  Model 


It  would  be  naive  to  think  that  all  major  military  systems 
are  developed  in  direct  conformity  with  the  approach  recommended 


2-11 


f 

i 


1 

I 

by  the  DOD.  Some  systems  are  the  direct  descendants  of  earlier 
ones,  and  thus  "land  running,"  so  to  speak.  Thus,  for  economic 
reasons  the  early  phases  in  the  acquisition  model  may  be  only 
perfunctorily  attended  to,  and  some  phases  may  be  compressed. 

Other  systems,  however,  evolve  almost  full-blown  as  the  synthesis 
of  various  minor  systems;  such  systems  are  not  subject  to  the 
requirements  for  major  ones.  The  danger  here  is  two-fold  and 
applies  to  both  human  factors  and  other,  strictly  technological, 
considerations.  First,  the  minor  systems  themselves  may  be 
ill-conceived.  Second,  a  composite  major  system  will  be  more 
than  simply  the  sum  of  its  components.  Fitting  together  small 
systems  to  produce  a  larger  one  should  not  be  attempted  without 
the  careful  planning  implicit  in  the  Circular  A-109  model. 

Finally,  it  is  worth  noting  that  recommendations  for  changes 
in  the  systems  acquisition  process  have  been  advocated  by  the 
Deputy  Secretary  of  Defense  in  a  recent  memorandum  (1981) .  He 
points  to  a  need  for  expediting  the  development  of  systems  by 
reducing  the  number  of  DOD- level  reviews  (DSARCS) ,  and  leaving 
more  of  the  key  decision-making  to  service-level  reviews  ((S)SARCS). 
The  MENS  would  be  shortened  substantially  in  length,  the  number 
of  milestones  would  be  reduced,  and  DSARCS  would  have  more 
representation  from  the  services.  In  general,  this  proposal 
would  decentralize  the  acquisition  process,  giving  the  DOD  less 
control  over  the  initiation  and  design  of  systems  and  leaving  the 
Secretary  of  Defense  with  less  influence  over  preliminary  planning. 


2-12 


The  Concept  of  the  Principal  Product 


As  stated  earlier,  the  principal  product  is  the  cumulative 
contribution  of  HP  to  the  operability  and  maintainability  of  a 
system.  Each  of  the  developmental  phases  in  the  acquisition 
cycle  yields  a  product;  each  product  has  distinctive  properties 
of  its  own  and  cumulative  effects  upon  the  products  of  subsequent 
phases.  A  comprehensive  discussion  of  the  concept  was  presented 
in  the  Phase  I  report,  and  the  necessary  basic  assumptions  and 
actions  can  be  found  in  Appendix  c  of  the  present  report. 

Exhibit  2-5  shows  the  principal  products  which  the  HF  staff 
should  contribute  in  each  of  the  development  phases.  A  summary 
of  these  (with  an  example  of  each)  is  presented  below,  followed 
by  a  discussion  of  additional  considerations  regarding  the 
principal  product  which  have  evolved  during  the  present  phase 
of  the  project. 

A  Brief  Discussion  and  Example  of 
Development  Cycle  Principal  Products 

A  discussion  of  the  phase-by- phase  principal  products  along 

with  a  simple  example  will  help  to  illustrate  both  the  importance 

and  nature  of  the  principal  product  concept.  A  running  example 
3 

involving  a  C  I  system  is  used  in  order  to  point  out  the  cumulative 
effect  of  HF-related  decisions  upon  the  nature  of  the  system. 

The  example  incorporates  considerations  which  have  emerged  from 
actual  systems,  and  it  reflects  the  same  role-of-man  issue 
utilized  in  the  case  study  of  the  maneuver  control  system  impact 
analysis  (Appendix  A) . 

Mission  Analysis  (Foie  of  Man) .  (See  Exhibit  2-6  for  an 
illustration  of  this  phase.)  In  general,  the  system  developer 
must  address  the  question  of  whether  or  not  the  proposed  system 
is  needed.  This  question  is  answered  through  a  threat  analysis. 
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Exhibit  2-6 

Human  Factor*  Principal  Product* 

-FIRST  PRASE  *  MISSION  ANALYSIS- 


INSTtCATION  ACTION  ONTCOMI 
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which  provides  the  basic  justification  in  the  MENS  and  helps 
to  determine  the  system  mission.  The  operational  conditions 
and  system  functions  are  then  identified,  and  a  parallel  process 
is  conducted  to  determine  the  functions  of  man  under  these  same 
conditions.  The  principal  product  is  the  Role-of-Man  Statement 
resulting  from  these  activities.  The  statement  should  include 
the  following  considerations: 

e  The  effects  of  alternative  system  concepts  upon  man 
(habitability,  safety,  etc.) 

e  The  relative  advantages  of  alternative  human  functions 
for  various  system  concepts. 

e  The  relative  disadvantages  of  alternative  human  functions 
for  various  system  concepts. 

e  The  required  human  performance  and  capabilities  for 
each  function. 

e  The  implications  of  each  alternative  system  concept  for 
training,  manpower,  life  support,  logistics,  etc.). 

e  A  list  of  human  factors  design  features  which  could 
facilitate  successful  system  operation  under  each 
alternative  concept. 

Example  -  The  example  is  intended  to  illustrate  the  fundamental 
characteristics  of  the  principal  products;  it  is  not  intended  to 
suggest  an  idealized  C^I  system  design.  Such  systems  are  too 
complex  and  varied  to  allow  such  an  undertaking.  The  system 
design  concept  to  be  analyzed  provides  that  the  nodes  within  the 
system  serve  as  receivers,  filters  and  transmitters.  Thus, 
information  from  the  field  flows  upward  through  hierarchies,  with 
the  nodes  at  each  level  reducing  the  information  such  that  it 
can  be  utilized  efficiently  by  the  end-user  at  division  command. 
Given  this  system  concept,  the  role  of  the  operator  must  be 


determined.  The  decision  is  to  employ  the  non-dedicated  user 
as  operator. 

The  non-dedicated,  or  "casual,”  user  idea  has  certain 
advantages.  First,  the  impact  of  battlefield  conditions  upon 
the  availability  and  performance  of  highly  trained  specialists 
will  be  minimized.  Thus,  personnel  with  different  backgrounds 
can  step  in  when  necessary.  Second,  such  an  approach  limits  the 
necessary  allocations  for  training.  Third,  the  projected  use  of 
non-dedicated  users  focuses  attention  upon  the  development  of 
software.  Therefore,  there  is  no  gray  area  between  training  and 
technology  which  can  create  ambiguity  for  systems  developers  in 
respect  to  their  allocation  of  resources. 

The  main  disadvantage  of  using  the  non-dedicated  user  is 
the  burden  placed  upon  automation.  A  great  deal  of  equipment 
and  software  is  necessary  to  drive  the  system  operation,  and 
thus,  mobility  may  be  restricted.  However,  in  the  present  case 
it  is  assumed  that  recent  advances  in  technology  will  provide 
the  capability  for  complex,  yet  compact,  automated  elements. 

Concept  Development  (Allocation  of  Functions) .  (See  Exhibit  2-7 
for  an  illustration  of  this  phase.)  The  system  developer  studies 
the  alternative  concepts  which  evolved  during  the  Mission  Analysis 
phase.  For  the  concept  selected,  the  HF  principal  product  is 
allocation  of  functions  to  man  and  machine.  This  should  be 
documented  in  the  DCP.  In  modern  systems  this  usually  means  a 
decision  about  the  degree  of  automation.  The  basic  steps  in  the 
allocation  process  are: 

e  Specify  the  human  factors  criteria  selected  for  allocation 
of  functions  (e.g.,  response  time,  error  rate,  cost, 
safety,  etc.). 
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Exhibit  2-7 

Human  Facton  Principal  Products 


-SECOND  PHASE:  CONCEPT  DEVELOPMENT  PHASE - 


STUATUN  ACTION  OUTCOME 


•  List  alternative  allocations  of  each  function  to: 

one  or  more  operators/maintainers ;  machine  only;  combination 
of  man/machine. 

•  Estimate  feasibility  for  alternative  allocations  of 
each  function,  considering  the  following: 

-  human  performance  capabilities  required 

-  machine  capabilities  required 

-  workload 

-  user  acceptance 
bottlenecks 

-  mission  impacts 

-  criticality  of  functions. 

•  Evaluate  the  alternative  allocations  of  each  function. 

All  allocations  of  functions  should  be  listed  in  a 
matrix  and  systematically  compared  with  the  criteria 
formulated  in  the  first  step. 

Example  -  In  the  Mission  Analysis  phase  the  role  of  man  selected 

3 

for  the  preferred  C  I  concept  was  that  of  the  non-dedicated 
user.  Given  that  the  chosen  concept  and  associated  role  of  man 
are  adhered  to  in  the  present  (Concept  Development)  phase,  the 
decision  tree  in  Exhibit  2-8  shows  that  the  general  range  of 
possible  man-machine  allocations  is  to  some  extent  predetermined. 


Exhibit  2-8 


Degree  of  Functions  Allocations  to  Man  and  Machine 
as  Consequence  of  Role  Determination 


Role  of  Man 


Degree  of 
Allocation 


High  Machine 
0 


Low  Operator 


Moderate-High 

Machine 

Moderate-  Low 
Operator 


Moderate-  Low 
Machine 

Moderate-High 

Operator 


Low  Machine 
High  Operator 


If  operators  are  to  be  non-dedicated  users,  mode rate- to- 

high  machine  allocation  is  required.  Thus,  while  the  criteria 

for  the  allocation  of  functions  will  be  about  the  same  as  those 
3 

for  most  C  I  systems  (e.g.,  information  bottlenecks,  response 
time,  cost,  etc.),  the  alternative  allocations  evaluated  against 
these  criteria  will  be  restricted  to  the  range  represented  by  the 
left-hand  side  of  the  scale,  a  range  reflecting  emphasis  upon 
machines  (automation) .  Within  this  range  the  feasibility  of 
alternative  allocations  must  be  determined.  For  example,  in  the 
area  of  user  acceptance  there  is  a  real  question  of  operator 
overload  in  C3I  systems.  Heavy  allocation  to  machines  will 
theoretically  provide  the  answer  to  this  problem,  but  may  frustrate 
the  needs  of  operators  to  perform  at  the  highest  skill  level. 

If  only  non-dedicated  users  are  employed,  this  should  not  be  a 
problem;  however,  the  possible  use  of  some  specialists  as  operators 
should  be  addressed  in  this  regard. 
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At  this  point  the  crux  of  the  human  factors  issue  emerges: 

What  if  the  Mission  Analysis  principal  product  (role  of  man  as 
non-dedicated  user)  had  not  been  derived?  System  functions 
might  have  been  allocated  in  such  a  way  that  successful  operation 
would  have  required  operator  expertise  far  exceeding  manpower 
levels  and  conflicting  with  realistic  combat  scenarios.  Assuming 
that  the  non-dedicated  user  is  the  best  role-of-man  alternative, 
the  failure  to  derive  this  product  initially  might  result  in 
functional  allocations  and  subsequent  training/HF  decisions  that 
conflict  with  efficient  system  performance. 

Demonstration/Validation  (Task  Analysis  &  Human  Factors  Engineerim 
Requirements) .  (See  Exhibit  2-9  for  an  illustration  of  this 
phase.)  The  general  purpose  of  the  effort  at  this  stage  is  to 
demonstrate  the  selected  system  concept  and  test  its  feasibility 
with  regard  to  mission  requirements.  Two  closely  linked  principal 
products  should  be  developed  in  this  phase:  task  analysis  and 
human  factors  engineering  requirements.  Given  the  allocation-of- 
f unctions  product  of  the  Concept  Development  phase,  the  following 
actions  should  be  taken  by  the  HF  staff  in  Demonstration/Validation 

•  Participate  in  construction  of  mock-ups. 

•  Conduct  task  analyses  (functional  analyses,  sequence 
analyses,  etc.) 

e  Derive  station  arrangement,  workspace,  console,  and  C/D 
concepts. 

e  Conduct  simulation/mock-up  evaluations. 

•  Determine  human  performance  and  HF  engineering  require¬ 
ments. 

•  Participate  in  prototype  development. 

e  Participate  in  DT/OT. 


Exhibit  2-9 

Human  Factors  Principal  Products 


-THIRD  PHASE:  OEMOHSTRATIOH  AHD  VAUDATIOH- 


INSTIGATION  ACTIO*  OUTCOME 


•  Perform  additional  task  analyses  on  prototype,  as  needed, 
e  Validate  HF  engineering  requirements. 

Task  analyses  should  be  performed  iteratively  on  successive 
refinements  of  the  mock-up  and  prototype  until  the  human  per¬ 
formance  aspects  of  the  system  are  within  the  capabilities  of 
the  projected  operators  and  maintainers.  The  HF  engineering 
requirements  will  be  derived  from  the  task  analytic  findings. 

Example  -  Given  that  operators  will  be  non-dedicated  users  and 
that  the  system  functions  are  allocated  largely  to  machines  in 
our  hypothetical  C3I  system,  the  HF  engineering  requirements 
will  be  critical  to  successful  operation.  Mock-ups  and  prototypes 
which  incorporate  advanced  interface  concepts  must  be  constructed 
so  that  personnel  who  are  not  highly  trained  in  the  use  of 
computerized  systems  can  use  the  consoles  and  keyboards.  A 
universal  symbology,  simple  prompts,  immediate  error  notification, 
and  data  entry  via  a  typewriter- style  keyboard  are  concepts  that 
might  be  tested.  The  estimated  rate  and  volume  of  message  flow 
throughout  the  system  should  drive  both  the  task  analyses  and 
formal  evaluations.  Also,  since  the  non-dedicated  user  may 
have  any  one  of  a  number  of  specialties  (e.g.,  planner,  analyst, 
runner,  etc.),  intertask  work  flow  might  be  analyzed. 

Full-Scale  Development  (Optimal  Man-Machine  Interface) .  (See 
Exhibit  2-10  for  an  illustration  of  this  phase.)  The  system 
developer  is  faced  with  the  decision  of  whether  or  not  to  accept 
or  modify  the  prototype  system  built  in  the  Demonstration/ 
Validation  phase.  For  HF  proper,  this  phase  entails  the  consider¬ 
ation  of  design  alternatives  in  respect  to  man-machine  interface. 
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Exhibit  2-10 

Human  Factors  Principal  Products 


-FOURTH  PHASE  s  FULL  SCALE  DEVELOPMENT- 


BATfOM  ACTION  OUTCOMI 


Essentially,  the  HF  staff  must  further  refine  and  evaluate 
interfaces,  which  will  entail  the  following  considerations: 

e  fulfillment  of  human  factors  requirements 

e  conformity  to  human  factors  design  criteria 

e  quantitative  measures  of  system  performance 

•  detection  of  undesirable  design  or  procedural  features. 

3 

Example  -  The  developers  of  the  proposed  C  I  system  will  require 
that  data  be  formatted  and  entered  by  operators  within  the  constraints 
of  various  environmental  conditions  and  specific  time,  and  accuracy 
requirements.  Due  to  proposed  engineering  changes  in  the  prototype, 
the  display  size  on  the  console  is  to  be  slightly  reduced.  The 
HF  staff,  therefore,  will  be  concerned  with  the  sensory/perceptual 
characteristics  of  the  smaller  display  which  might  affect  length, 
of  lines,  number  of  lines,  menu  size,  etc.;  these  are  factors 
that  could  appreciably  influence  operator  performance.  Thus,  the 
final  human  factors  effort  must  involve  a  fine  tuning  of  the 
interface  characteristics  of  the  display  in  light  of  these  engineering 
design  changes  and  refinements. 

In  summary,  both  the  importance  and  the  interdependence  of 
the  principal  products  for  the  different  acquistion  phases 

i 

should  be  clear  by  this  point.  The  initial  role-of-man  analysis 
and  following  allocation  of  functions  provide  the  basic  direction 
for  task  analysis  and  HF  engineering  requirements  and,  finally, 
man-machine  interfaces. 

Should  the  first  step  be  incomplete,  the  entire  human  factors 
effort  will  be  in  jeopardy.  Too  many  systems  either  have  been 
failures  or  have  required  costly  redesign  for  precisely  this 


reason 


Additional  Comments  on  the  Principal  Product  Concept 

Both  the  conduct  of  the  case  studies  and  extensive  contact 
with  personnel  representing  HF  labs  and  systems  development 
organizations  have  led  to  two  primary  conclusions  concerning  the 
principal  products:  (1)  the  need  for  the  concept  is  more  urgent 
than  ever,  and  (2)  there  are  currently  numerous  problems  regarding 
the  proper  implementation  of  the  principal  product  in  systems 
development.  With  respect  to  the  first  point,  the  increasing 
need  for  this  concept  stems  primarily  from  the  effects  of  rapidly 
advancing  technology  upon  the  complexity  of  present-day  systems. 
This  fact  has  been  made  clear  in  both  the  present  and  earlier 
reports.  Regarding  the  question  of  principal -product  implemen¬ 
tation,  a  number  of  observations  have  been  made: 

1.  HF  personnel  often  have  little  or  no  voice  in  the 
Mission  Analysis  proceedings,  in  which  the  critical 
Role-of-Man  analysis  should  be  conducted. 

2.  Human  factors  personnel  serve  infrequently  on  DSARC/ 
(S)SARC  panels,  so  that,  consequently,  these  panels 
rarely  confront  human  factors  issues. 

3.  HF  decisions  often  are  informal  and  difficult  to  capture. 
A  thorough  documentation  procedure  is  necessary  if  HF 
laboratories  and  departments  are  to  represent  principal 
products  as  real  contributions  to  system  design. 

4.  Since  some  systems  are  relatively  minor  variations  of 
earlier  ones,  for  purposes  of  economy  the  new  system 

may  essentially  begin  development  late  in  the  acquisition 
cycle  (e.g.,  at  the  Demonstration/Validation  stage).  The 
principal  products  of  predecessor  systems  in  such  cases 
must  be  recognized  as  the  sources  to  be  considered  in 
the  development  of  the  new  system. 


5.  Since  systems  are  extremely  variable  from  several 
standpoints  (configuration,  size,  mission,  degree  of 
automation,  etc.),  there  exists  a  need  for  a  tool  that 
can  be  used  to  link  the  principal  product  concept  to 
system- specific  HF  engineering  issues. 

The  first  three  observations  reflect  the  need  for  expanding 
the  role  of  the  HF  practitioner,  a  need  discussed  in  the  earlier 
interim  report  as  well  as  in  the  present  report.  However, 
points  (4)  and  (5)  require  a  bit  more  elaboration.  First,  since 
many  systems  are  variations  of  earlier  ones,  only  a  clear  under¬ 
standing  of  the  predecessor,  or  "reference,"  system  HF  data  and 
issues  can  prevent  the  recurrence  of  prior  human  factors  defi¬ 
ciencies.  This  is  especially  important  in  cases  in  which  there 
are  organizational  pressures  to  incorporate  as  much  of  the 
reference  system (s)  as  possible  into  the  new  one. 

In  regard  to  the  question  of  directly  linking  the  principal 
product  notion  to  system- specific  HF  issues  and  problems,  a 
taxonomic  approach,  such  as  that  shown  in  Exhibit  2-11  could 
serve  as  a  first  step  by  illuminating  the  HF  engineering 
commonalities  and  differences  of  various  systems.  The  taxonomy 
could  also  serve  as  the  organizing  principle  for  a  second  step, 
the  creation  of  a  human  factors  data  base.  This  could  be  used 
to  consolidate  into  a  principal  product  reference  system  the 
human  performance  functions  and  HF  actions/issues  which  have 
emerged  during  the  acquisition  of  different  systems. 

As  a  final  note,  it  should  be  clear  that  the  principal 
product  is  not  intended  to  be  a  rigid  specification.  The 
principal  product  is  both  a  process  and  a  concept.  As  a  process, 
it  embodies  the  kinds  of  HF  considerations  which  must  come  into 
play  at  the  various  stages  of  system  development.  As  a  concept. 
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it  is  a  focal  point  around  which  come  together  al 
of  the  human  factors  community  regarding  the  perf 
characteristics  of  military  systems. 


CHAPTER  3 

THE  COST-BENEFIT  APPROACH:  IMPACT  ASSESSMENT 

Background 

There  is  a  basic  generic  affinity  between  the  concept  of 
cost-benefit  analysis  and  the  concept  of  human  factors  engineering. 
Both  were  originally  conceived  as  means  to  prevent  or  minimize 
the  commission  of  gross  errors.  There  is  an  even  broader  affinity: 
Cost-benefit  analysis  is  based  on  economics,  and  human  factors 
engineering  is  a  part  of  the  larger  engineering  enterprise. 

Both  economics  and  engineering  focus  on  return  on  invested 
resource  (ROI) .  In  economics,  the  resource  is  money  or  its 
equivalent;  in  engineering  the  resource  is  likely  to  be  time  or 
energy  or  both.  Insofar  as  time  and  energy  can  be  converted  to 
dollars,  both  fields  have  a  common  objective:  efficiency. 

Given  these  commonalities,  the  idea  of  evaluating  the 
contribution  of  HF  to  military  system  developments  is  an  appealing 
one,  particularly  in  the  sense  that  the  "efficiency"  of  the 
development  process  could  be  enhanced  as  well  as  the  "efficiency” 
of  the  resultant  system. 

This  study  recommends  and  develops  a  conceptual  methodology 
for  using  one  form  of  the  cost-benefit  approach  to  assess  the 
contributions  of  human  factors  in  military  system  development. 

As  stated  earlier,  the  major  components  of  the  conceptual 
methodology  developed  during  Phase  I  are: 

•  A  rationale  for  human  factors  considerations  in  system 
development  with  specific  analyses  for  Human  Factors 
Principal  Products  during  the  major  development  milestones 
and  other  system  specific  efforts  and  technology-base 
issues  (Chapter  2) . 


A  multi-step  Impact  Assessment  Framework  for  formally 
measuring  and  relating  human  factors  contributions  to 
military  system:  cost  control,  capability,  and  compat¬ 
ibility. 

.  Integration  of  HF  Principal  Products 
and  Impact  Assessment 

The  human  factors  principal  products  for  each  major  phase 
of  the  system  acquisition  were  described  in  Chapter  2  and  are 
summarized  in  Exhibit  2-5.  Their  importance  to  the  impact 
assessment  implementation  will  now  be  discussed. 

The  principal  products  from  each  phase  of  the  system 
acquisition  are  a  meaningful  way  to  represent  the  scope  of 
human  factors  in  a  military  system  development.  These  phased 
products  are  intended  to  vary  in  content  and  specificity  from 
the  very  conceptual  requirement  level  to  the  very  detailed 
design  level,  just  as  is  the  case  with  products  of  systems 
engineering,  logistics,  etc.  during  each  phase.  A  detailed 
description  of  the  principal  products  is  presented  in  Appendix  C 

In  general,  each  HF  principal  product  will  include: 

e  A  check-off  list  of  critical  HF  issues  tailored  to  the 
phase  of  system  development.  Past  experience  and 
documentation  on  reference  systems  or  functions  and 
new  technology  are  major  inputs  to  the  check-off  list. 

e  Empirical  and/or  analytical  findings  from  HF  analysis, 
design,  test,  and  evaluation  techniques  carried  out 
during  the  phase  of  system  development.  Examples  of 
HF  techniques  include:  mission  profile/scenario 
analysis,  function  flow  diagrams,  decision/action 
diagrams,  task  descriptions,  etc. 


•  A  specific  set  of  recommended  HF  actions  that  are 
defined  in  terms  of  system-  and  personnel- related 
empirical  measures  (e.g.,  firing  rate,  mean  time  to 
repair,  human  engineering  deficiencies,  etc.) . 

•  A  preliminary  translation  of  the  actions  in  terms  of 
human  factor  and  system  engineering  metrics  and  their 
expected  primary  impact  areas.  This  translation  will 
normally  be  a  narrative  discussion. 

e  An  HF  management  plan  update  for  the  remainder  of  the 
current  phase  or  for  the  next  phase  of  the  system 
development. 

In  many  instances  the  analysis  of  the  human  factors  actions 
can  stop  at  this  point  and  will  not  require  a  formal  impact 
assessment.  For  example,  if  the  benefits  of  an  action  are 
self-evident  (e.g.,  safety)  and  the  expenditure  to  achieve  the 
benefits  is  within  a  program  manager's  selected  budget  threshold, 
then  a  formal  benefit-cost  impact  assessment  is  not  essential. 

On  the  other  hand,  for  those  human  factors  actions  which 
have  substantial  input  and  output  uncertainty,  resource  allocations 
competition,  high  visibility,  or  simply  need  an  analytical 
demonstration  of  their  worth,  the  10-step  impact  assessment 
framework  shown  in  Exhibit  3-1  can  be  used  to  translate  quanti¬ 
tatively  the  Expected  impacts  in  terms  of  system-mission  cost, 
capability,  and  compatibility.  A  description  of  each  of  the  10 
steps  is  provided  in  Appendix  D  to  this  report. 


Linking  Human  Factors  Changes  to  System 
Cost,  Capability,  and  Compatibility 

As  noted  previously,  HF  is  an  activity  directed  toward'-the^ 
goal  of  system  efficiency  along  a  major  route  that  can  be  labeled 
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Impact  Assessment  Framework 


"the  prevention  of  gross  errors."  Most  of  the  criticisms  of 
contemporary  military  systems  reveal  that  gross  errors  can  be 
made  and  are  made  in  the  decisions  among  design  alternatives. 

One  of  the  reasons  that  it  is  always  difficult  to  prove 
the  value  of  HF  in  a  syllogistic  fashion  is  that  the  absence  of 
error  can  never  be  tied  to  a  single  cause.  The  logic  has 
always  had  to  be  of  the  form:  No  HF  was  done;  a  design  error 
was  made;  hence  it  is  conceivable  that  the  design  error  would 
not  have  been  made  had  HF  been  done.  This  type  of  reasoning  is 
not  very  powerful  or  convincing. 

A  related  weakness  stems  from  the  fact  that  there  appear 
to  be  some  systems  that  work  rather  well  (are  efficient) ,  and 
upon  which  little  or  no  official  HF  was  performed.  Engineering 
folklore  can  be  read  to  suggest  that  a  reasonably  experienced 
or  alert  PM  has  learned  some  rudimentary  HF  and  can  be  his/her 
own  HF  specialist  by  the  routine  exercise  of  a  modicum  of 
"common  sense."  This  approach,  however,  has  usually  only  been 
effective  when  "lessons  learned"  from  a  similar  or  predecessor 
system  are  available. 

The  negative  folklore  cannot  be  entirely  refuted  by  the 
logic  of  negative  cases.  Consequently,  there  has  arisen  a 
growing  sense  that  more  powerful  means  of  persuasion  were 
required. 

Our  basic  position  is  that  the  combination  of  the  concepts 
of  human  factors  principal  products  and  that  version  of  cost- 
benefit  known  as  impact  analysis  could  constitute  such  a  more 
powerful  means. 
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One  advantage  provided  by  this  combination  is  that  it 
tightly  ties  what  the  HF  practitioners  do  in  the  system 
development  process  with  what  the  other  principal  participants 
are  doing.  These  links  are  manifested  in  the  first  instance  by 
a  co-adherence  to  the  schedule-of-events  in  system  development — 
regardless  of  whether  that  schedule  is  the  official  DOD  version 
or  some  ad  hoc  variation.  Secondly,  the  principal  product  idea 
leads  to  a  tangible  HF  contribution,  the  substance  and  timing 
of  which  are  predictable.  In  other  words,  the  managers  of  the 
system  development  process  can  be  given  specific  expectations 
about  what  the  HF  practitioners  are  doing  to  facilitate  the 
system  development  process.  If  necessary,  such  managers  can 
then  make  an  equivocal  albeit  subjective  judgement  on  the  value 
of  the  contribution.  They  can  respond  to  the  narrow  but  realistic 
question:  Were  my  expectations  met? 

The  framework  of  impact  analysis  permits  that  same  judgement 
to  be  made  in  both  a  more  sophisticated  and  a  more  objective 
manner.  Impact  analysis  provides  a  way  of  asking:  What  would 
have  been  the  consequence  of  not  having  the  HF  product?  What 
could  the  wrong  decision  have  cost? 

Linkage  Between  HF  Principal  Products  and 
the  Impact  Assessment  Framework 

The  basic  relationship  between  the  HF  principal  product 
recommended  actions  and  the  impact  assessment  framework  is 
illustrated  in  Exhibit  3-2.  A  brief  description  of  the  formal 
linkage  between  the  human  factors  principal  product  recommendations 
to  change  a  system  design  and  the  implications  of  that  change 
for  the  cost,  capability,  and  compatibility  of  the  system  is 
given  next. 
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The  linkage  between  the  human  factors  empirical  findings 
and  the  impact  areas  consists  of  two  components  (depicted  by 
and  L2  in  Exhibit  3-2) .  A  simplified  illustration  of  the 
process  is  shown  below. 

L1  l2 

Human  Factors  Human  Factors  and  System-Mission 

Empirical  Findings  System  Engineering  Impact  Areas 

and  Recommendations  Metrics 

Linkage  L1  within  the  scope  of  the  HF  principal  product,  and 
linkage  L2  is  comprised  of  the  impact  assessment  methodology. 

On  the  left  side  of  Exhibit  3-2,  a  typical  list  of  human 
factors  analysis  activities  is  shown.  Based  on  these  analyses, 
a  number  of  specific  deficiencies  and  recommended  changes  are 
determined  about  the  role  of  man,  his  allocated  functions,  his 
tasks  and  human  engineering  requirements,  and  the  man-machine 
interfaces  over  the  system  development-acquisition  cycle. 

These  empirically  based  measures  are  then  translated,  via 
linkage  L^,  into  a  set  of  common  or  related  human  factors  and 
engineering  metrics. 

The  metrics  are  depicted  as  cells  within  a  triangle  (in 
the  center  of  Exhibit  3-2) .  The  metrics  are  related  formally 
to  the  system-mission  impact  areas  of  Cost,  Capability,  and 
Compatibility  via  linkage  L2,  the  impact  assessment  methodology. 
The  location  of  the  metrics  in  the  cells  within  the  triangle 
indicates  an  expected  first-order  relationship  with  the  impact 
areas.  Only  a  few  example  metrics  are  shown  in  the  exhibit. 

The  diamond- shaped  cells  contain  metrics  that  are  common  to  two 
impact  areas.  For  example,  the  metric  reliability  is  shown  in 
Exhibit  3-2  as  having  a  first-order  association  with  impact 


areas  Cost  and  Capability  because  it  is  located  within  the 
diamond- shaped  cell  that  is  defined  by  the  intersection  of  the 
Cost  and  capability  diagonal  columns.  Metrics  within  a  triangle¬ 
shaped  cell  are  expected  to  have  a  primary  association  with  the 
impact  area  indicated  above  the  cell.  For  example,  the  metric 
Operation  and  Support  Cost  is  shown  in  Exhibit  3-2  as  having  a 
primary  association  with  the  Cost  impact  area. 

When  a  human  factors  change  involves  all  three  impact 
areas  the  linkage  would  be  modeled  in  terms  of  a  combination  of 
specific  metrics  from  two  or  more  cells.  Let  us  take  an  example, 
in  which  a  human  factors  change  is  mapped  onto  the  metric  Crew 
Accommodations,  which  has  a  first-order  association  with  the 
Cost  and  Compatibility  impact  areas  (it  could  also  be  related 
to  the  capability  impact  area)  via  a  metric-metric  relationship 
such  as:  Crew  Accomodations  to  Task  Loading.  The  latter 
metric  is  in  a  cell  in  Exhibit  3-2  related  to  the  Capability 
metric  area.  A  preliminary  list  of  metrics,  based  on  our 
findings  in  the  Phase  I  portion  of  this  study  is  presented  in 
Appendix  E.  Eventually,  a  comprehensive  set  of  metric  terms 
that  will  provide  a  standard,  common  vocabulary  for  human 
factors  and  systems  engineering  practitioners  should  be  developed. 

Using  the  Impact  Assessment  Methodology 

Linkage  L2  consists  of  the  impact  assessment  process.  For 
those  HF  actions  that  require  formal  quantification  of  their 
costs  and  benefits,  the  10  steps  listed  in  Exhibit  3-1  would 
be  carried  out. 

The  impact  assessment  methodology  is  designed  to  assist 
program  managers  who  would  use  analytical  help  or  products  in 
their  HF-related  decisions,  and  to  offer  guidance  to  those  who 


would  provide  the  analytical  help.  A  fundamental  aim  is  to 
provide  a  tool  for  achieving  parity  in  human  factors  partici¬ 
pation  in  military  system  development  programs. 

Toward  those  ends,  the  impact  assessment  methodology  can 
be  used  in  the  following  ways: 

e  As  a  Discipline  -  to  ensure  that  essential  steps  are 

carried  out  completely  and  consistently  across  different 
impact  assessments.  The  methodology  will  help  to  organize 
materials,  direct  attention  to  the  proper  issues,  and 
demonstrate  that  impact  assessments  are  not  limited 
only  to  issues  that  are  quantified  in  terms  of  dollars. 

e  To  Compare  Alternatives  and  Select  the  Preferred  One- 
in  terms  of  their  mission-system  cost,  capability,  and 
compatibility  values. 

e  To  Formally  Introduce  HP  Parameters  into  the  Design 
Process  along  with  Cost  and  Capability-for  consider¬ 
ation  in  the  design  tradeoff  decisions. 

The  question  of  when  an  impact  assessment  should  be  initiated 
does  not  have  a  simple,  cookbook  answer.  Instead,  the  answer  is 
dependent  upon  a  number  of  contingencies,  many  of  which  involve 
political,  judgemental,  or  intuitive  considerations.  The  key 
decision  issues  appear  to  be:  (1)  Is  a  quantitative  interpre¬ 
tation  required  for  the  decision-maker  to  assess  effectively  the 
recommended  HF  actions?  (2)  Is  the  problem  amenable  to  a  formal 
impact  assessment  (that  is,  can  an  impact  assessment  provide  the 
required  insights  and  precision)?  and  (3)  Can  the  impact  assess¬ 
ment  be  done  within  reasonable  time  and  resource  constraints 
(including  available  data,  expertise,  and  models)?  These  three 
issues  are  interdependent,  particularly  (2)  and  (3) . 
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In  an  attempt  to  gain  insight  into  the  practicality  of 
applying  the  impact  assessment  methodology  to  HF  actions,  two 
case  studies  were  initiated:  the  F-18  (Navy)  and  a  hypothetical 
Command/Control  System  (based  on  TOS/SIGMA  for  the  Army) . 

Each  case  study  entailed  developing  an  HF  principal  product 
and  applying  the  impact  assessment  methodology  to  a  particular 
HF-related  action.  The  documentation  for  each  of  the  case 
studies  is  included  in  Appendix  A,  and  the  findings  from  these 
studies  are  discussed  in  Chapter  4. 
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CHAPTER  4 
CONCLUSIONS 

This  chapter  contains  findings  and  recommendations .  The 
findings  address  the  following  questions! 

(1)  Are  the  conclusions  of  the  Phase  I  effort  still  valid 
after  the  case  study  exercise? 

(2)  Is  the  HP  Principal  Products  and  Impact  Assessment 
Framework  (PPIAF)  approach  applicable  to  the  evaluation 
of  HF  contributions  to  system  development? 

(3)  If  applicable,  is  the  Human  Factors  PPIAF  approach 
practical  and  worth  the  effort?  This  question  is 
particularly  important  to  the  impact  assessment  process. 

(4)  Does  the  Human  Factors  PPIAF  approach  call  for  a  redefinition 
of  the  role  of  the  HF  practitioner? 

In  addition,  the  major  constraints  experienced  during  the  case 
studies  are  noted. 

The  recommendations  present  several  action  items  that  are 
essential  to  improve  the  state  of  the  art  in  the  implementation 
of  the  Human  Factors  PPIAF  approach. 

Conclusions  of  the  Phase  I  Study 

The  Phase  I  effort  concluded  that: 

e  A  conceptual  basis  for  relating  HF  contributions  can 
be  defined. 

•  The  HF  contributions  are  measurable . „ 

e  A  methodology  for  evaluating  HF  impacts  is  feasible. 


Our  Phase  II  findings  are  that  these  statements  are  valid,  but 
that  there  are  important,  pragmatic  qualifications  regarding  their 
applicability  and  practicality.  The  HF  principal  product  outline 
and  impact  assessment  methodology  presented  in  Phase  I  are  some¬ 
what  ideal.  During  Phase  II,  those  concepts  were  made  considerably 
more  realistic  from  the  standpoint  of  their  implementations  and 
interrelationships.  We  also  learned  that  there  are  several  con¬ 
straints  that  the  HF  principal  products  and  impact  assessment 
methodology  must  and  can  deal  with.  These  constraints  are 
discussed  next,  along  with  the  issue  of  practicality. 

Applicability  and  Practicality  of  the 
Human  Factors  PPIAF  Approach 

The  case  studies  provide  enough  of  an  experimental  basis 
to  allow  the  conclusion  that  both  the  HF  principal  products  and 
the  impact  assessment  methodology  are  applicable  to  the  identifi¬ 
cation  and  analysis  of  HF  issues  in  military  system  design. 

They  provide  a  logical  and  substantive  improvement  over  the 
currently  constrained  methods  for  performance  and  evaluation  of 
human  factors  in  the  context  of  military  system  development. 

The  question  of  practicality  cannot  be  answered  so 
unequivocally,  in  part  because  the  case  studies  were  carried 
out  not  in  situ  but  rather  post  hoc,  and  in  part  because  the 
case  studies  do  not  represent  the  full  spectrum  of  HF-system 
development  combinations  likely  to  be  incurred  in  reality.  We 
can  conclude,  that  unless  a  system  development  program  has  been 
carefully  documented,  that  it  is  impractical  to  attempt  an 
HF  principal  product  or  impact  assessment  analysis  in  a  post 
hoc  setting.  The  major  reason  for  the  proviso  is  the  lack  of 
relevant,  valid  data.  Also,  because  of  the  many  hypothetical 


premises  necessitated  **y  a  lack  of  data  in  the  case  examples, 
we  could  not  empirically  investigate  whether  impact  assessments 
tend  to  provide  results  that  are  any  different  than  those 
provided  by  subjective  arguments  supporting  interim  recommendations 
at,  say,  stage  in  Exhibit  3-2.  Obviously  there  will  be  a 
set  of  cases  for  which  subjective  arguments  are  sufficient  and 
appropriate.  However,  when  there  is  high  uncertainty,  resource 
allocation  competition,  high  visibility  considerations,  and 
analytic  support  for  other  candidate  and  non-HF  actions,  then 
the  use  of  the  impact  assessment  methodology  is  in  our  judgment 
an  appropriate  and  pragmatic  decision.  With  regard  to  the 
impact  assessment  methodology  data  and  model  requirements,  we 
find  that  since  it  builds  upon  the  Life  Cycle  Cost  and  Integrated 
Logistics  Support  analyses  called  for  in  current  DOD  directives 
and  instructions  for  major  system  developments,  it  is  no  less 
feasible  than  they  are.  However,  these  types  of  analyses  do 
require  some  expertise  that  human  factors  practitioners  might 
not  have.  All  of  these  analytical  methodologies  are  most 
practical  if  they  are  implemented  during  the  system  development 
process. 

Though  the  HF  impact  assessment  methodology  is  designed  as 
an  independent  activity,  it  is  intended  to  chronologically 
follow  the  development  of  the  HF  principal  products.  The  principal 
products  provide  essential  inputs  to  the  first  five  steps  of 
the  impact  assessment  methodology  (See  Exhibit  3-1) . 

There  appear  to  be  two  major  constraints  on  the  successful 
performance  of  an  HF  action  impact  assessment.  The  first  and 
paramount  are  data  constraints,  and  the  second  are  methodological 
difficulties.  Each  of  these  is  briefly  discussed  below. 
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Data  Constraints 


Data  constraints  include  problems  that  limit  the  impact 
assessment  analysis  because  data  are: 

e  insufficient  (or  non-existent) 

e  unvalidated 

e  inconsistent 

•  non-retrievable  (or  at  least  not  conveniently) . 

Thouqh  the  lack  of  adequate  data  was  the  most  severe  constraint 
experienced  in  the  case  studies,  it  is  a  problem  that  can  be 
treated  in  a  relatively  straightforward  manner.  What  is  needed 
is  a  deliberate  and  consistent  documentation  and  storage  process 
such  as  is  called  for  in  the  recommendations  at  the  end  of  this 
chapter. 

Methodological  Difficulties 

These  require  additional,  sometimes  significant,  effort 
and  expertise  to  be  dealt  with  properly.  Several  of  the  more 
prevalent  methodological  difficulties  are: 

1.  Isolating  Human  Factors  Impacts.  It  is  very  difficult — 
and  frequently  impossible — to  accurately  isolate  the  individual 
impacts  from  aggregated  impacts  when  the  human  factor  impacts 
are  not  independent  of  one  another,  or  when  the  individual  con¬ 
tributions  to  the  overall,  aggregated  impacts  are  not  individually 
measurable. 

In  such  instances,  it  is  necessary  to  aggregate  all  the 
concurrent  human  factors- related  actions.  When  the  impacts 
from  the  aggregated  human  factors  actions  cannot  be  distinguished 
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accurately  from  the  impacts  of  the  non- human  factors  actions , 
an  approximate  attribution  of  the  total  negative  and  positive 
impacts  on  the  military  system  to  the  contributing  actions  is 
required.  A  conceptual  basis  for  such  attributions  can  be 
found,  for  example,  in  Saaty  (1979)  and  Ostrofsky  (1977) .  It 
should  be  noted,  however,  that  isolation  of  HE  impacts  should 
not  be  so  difficult  in  a  system  under  development  where  HF 
practitioners  were  permitted  to  "practice"  the  PPIAF  approach. 

2.  Utilizing  Sophisticated  Techniques.  Many  of  the 
models  that  can  incorporate  intangible  impacts  are  difficult  to 
use  and  understand.  When  a  complex  procedure  is  needed  to 
assess  the  causal  relationship (s)  between  an  action  and  an 
impact,  it  will  often  be  necessary  to  employ  analytical  specialists 
to  apply  the  technique  and  interpret  the  results.  The  resources 
needed  to  do  the  analysis  are  part  of  the  cost-impact  assessment 
decisions . 

3.  component  vs.  System  Impacts.  Often  the  focus  of  the 
human  factors  R&D  activity  will  be  on  an  individual  procedure 
or  component,  and  not  an  entire  system.  When  the  procedure  or 
component  is  changed  as  a  consequence  of  the  human  factors 
related  actions,  the  impact  should  be  related  to  the  system's 
mission  capability,  cost,  or  compatibility.  It  is  often 
difficult  to  relate  the  results  of  an  analysis  of  a  part  to  the 
whole.  In  many  such  instances,  an  opportunity  cost  argument 
for  the  "freed"  resources  or  improved  capability  is  the  most 
appropriate  explanation  of  the  impact. 

4.  Tracking  Impacts  from  Phase  to  Phase.  The  conceptual 
process,  as  envisioned,  calls  for  the  consideration  and  assess¬ 
ment  of  human  factors  impacts  throughout  the  development  phases 

of  a  military  system.  Each  phase  represents  a  window  of  opportunity 
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for  human  factors- related  actions.  The  impact  assessment 
framework  is  intended  to  be  applied  to  the  candidate  actions 
within  each  phase.  In  keeping  with  the  baseline  concept  used 
in  cost- benefit  analysis,  the  projected  impacts  are  evaluated 
relative  to  a  specified  baseline.  When  design  or  procedure  is 

i 

changed,  the  baseline  for  subsequent  impact  assessments  is  also 
changed.  Consequently,  the  baseline  will  be  continually  updated 
as  changes  are  introduced  over  the  system  development  phases. 
Thus,  an  impact  forecasted  in  one  phase  will  not  necessarily  be 
additive  with  impact  forecasted  (claimed)  in  earlier  or  sub¬ 
sequent  phases.  Impacts  forecasted  should  be  presented  and 
documented  relative  to  the  baseline  for  the  phase  in  which 
they  are  generated,  and  not  casually  aggregated  across  phases. 

5.  Differentiating  R&D  Funding  Impacts.  In  general,  it 
will  not  be  apparent  how  to  relate,  in  a  quantitative  and 
precise  manner,  the  different  R&D  categories  used  to  fund  human 
factors  analyses  to  the  resulting  impacts  on  the  system  design. 
To  the  extent  that  the  R&D  budget  categories  and  the  type  of 
R&D  activity  are  defined  and  applied  in  a  consistent  manner, 
then  a  degree  of  differentiation  will  be  feasible. 

6.  System  vs.  Non-System  Specific  Impacts.  In  general, 
it  will  not  be  apparent  how  to  estimate  in  a  rigorous  way  the 
impacts  of  human  factors  research  beyond  a  specific  weapon 
system  setting — that  is,  to  classes  of  equipment,  or  to  general 
military  procedures.  This  is  particularly  true  for  "basic" 
research.  (Note:  This  problem  could  be  an  artifact  of  the 
budgeting  procedures  used  in  DOD.  a  distinction  between  human 
factors  research  (which  is  non-system  specific)  and  human 
factors  engineering  (which  is  system  specific)  might  remove  the 
problem  altogether. 
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7.  Risk  and  Uncertainty.  In  general,  the  treatment  of 
risk  and  uncertainty  in  models  that  assess  impacts  is  not 
adequate.  Procedures  do  exist  to  quantify  and  incorporate  risk 
in  cost  and  benefit  projections.  See,  for  example,  Fisher 
(1973),  Beers  (1957),  Sobel  (1965),  Murphy  (1970),  and  Dienemann 
(1966) . 


8.  Manpower  Policy.  Analytic  techniques  tend  to  mask  the 
military  manpower  policy  effects  on  candidate  design  changes 
generated  by  R&D  results  or  design  variations.  In  general,  a 
simulation  model  is  required  to  incorporate  the  impact  changes 
and  manpower  policy  requirements  in  a  consistent  framework. 

Such  models  are  often  not  applicable  until  the  later  stages  of 
the  system  development  process. 

9.  Rigor  vs.  Broad-Based  Analysis.  A  fundamental  issue 
underlying  many  of  the  above  points  is  whether  the  analysis 
should  be  primarily  rigorous,  and  statistically  complete,  or 
primarily  relevant  (descriptive  and  broad) .  A  rigorous  eval¬ 
uation  requires  (a)  formal  problem  statements,  (b)  definition 
of  the  analysis  and  testing  process  within  a  communicable  model 
framework*,  '(c)  the  capacity  for  replication  by  different  analysts 
at  different  times,  (d)  evaluation  designs  dependent  upon  the 
use  or  availability  of  baseline  or  control  groups,  and  (e)  that 
the  number  of  observations  and  the  number  of  model  relationships 
are  both  greater  than  the  number  of  test  characteristics  or 
variables  of  interest. 

The  notion  of  broad,  relevant  studies  is  used  here  to 
imply  a  broad-based  analysis  where  the  intent  is  to  describe 
what  has  taken  place  or  is  expected,  to  identify  tfre  predominant 
issues  in  a  certain  setting,  and  to  incorporate  them.  Many 
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relevant  variables  cannot  be  measured  in  a  rigorous,  quantifiable 
manner  (for  example,  user  acceptance  and  variations  in  skill- 
mix)  . 

This  dichotomy,  although  somewhat  contrived,  is  pertinent 
to  the  definition  of  the  cost-benefit  or  impact  analysis. 

This  is  so  because  not  all  human  factors  issues  or  parameters 
can  be  analyzed  in  a  rigorous  manner.  This  limitation  on 
rigorous  analysis  must  be  dealt  with  explicitly  in  a  tradeoff 
decision  during  the  formulation  step  of  the  analysis. 

Dealing  with  the  foregoing  methodological  limitations  in 
itself  requires  management  and  analysis  resources.  It  is 
important  to  recognize  what  the  related  estimated  costs  are  for 
the  evaluation  of  the  impacts.  If  the  costs  of  the  analyses 
are  comparable  to  the  expected  value  of  the  impacts,  then  it 
is  likely  that  the  analysis  as  defined  is  inappropriate  and  a 
simpler  approach  is  called  for. 

Redefinition  of  the  Role  of  the 
Human  Factors  Practitioner 

As  the  present  project  has  evolved,  it  has  become  increasingly 
apparent  that  the  role  of  HF  staff  engaged  in  major  system 
development  programs  needs  to  be  updated  and  clarified.  As 
suggested  in  chapter  2,  such  refinements  are  needed  to  ensure 
compliance  with  newly  issued  regulations.  But  there  has  also 
been  a  realization  that  a  strictly  formal  response  to  the  new 
regulations  would  not  suffice.  Rather,  a  review  and  adjustment 

*The  trade-offs  implicit  in  this  dichotomy  have  been  recognized 
for  many  years  and  were  addressed  forcefully  and  cogently  by  Sinaiko 
and  Belden  in  1963. 


is  needed  that  also  makes  explicit  the  observation  that  the  HF 
staff  has  a  dual  function,  both  parts  of  which  change  as  each  new 
stage  of  system  development  is  reached. 

One  part  of  the  dual  function  is  strictly  technical,  in  the 
sense  that  it  focuses  on  the  design  of  the  system.  In  this 
role  the  HF  staff  member  is  obliged  to  contribute  technical 
advice  on  specific  decisions.  This  role  is  more  or  less  well- 
delineated  in  the  literature  and  traditions  of  human  factors, 
and  is  also  reflected  in  the  wording  of  the  new  regulations. 

The  second  role,  however,  is  rarely  addressed  explicitly 
in  either  the  documentation  of  system  development  or  in  the 
texts  that  cover  HF  as  a  discipline.  This  second  role  is  the 
support  of  the  coherence  and  credibility  of  the  system  development 
program  as  a  whole.  In  other  words,  we  are  suggesting  that  the 
HF  staff  members  accept  a  significant  part  of  the  responsibility 
for  sustaining  the  viability  of  any  system  development  program 
to  which  they  are  assigned  in  a  professional  capacity.  The 
HF  professional  must  contribute  his  technical  knowledge  and 
expertise  to  the  PM,  as  well  as  understand  the  problems  and 
perspectives  of  the  other  disciplines  on  the  PM's  staff. 

The  Focus  Provided  by  the  Principal  Product 

Our  effort  in  deriving  the  principal  product  notion  has 
made  clear  the  fact  that  human  factors  should  be  conducted  by 
an  HF  staff  having  a  broad  understanding  of  both  the  systems 
acquisition  cycle,  as  outlined  in  DOD  directives,  and  the 
acquisition  process  in  actual  practice.  Bach  development  phase 
has  its  various  task  forces,  agencies,  objectives,  and  evaluation 
committees  associated  with  it.  In  order  to  ensure  that  human 
factors  has  maximum  impact  upon  the  development  of  a  system, 

HF  personnel  must  be  able  to  influence  those  parties  who  make 
critical  decisions  during  the  development  cycle. 


When  HF  expertise  contributes  to  such  decisions,  the 
resultant  human  factors  requirements  have  a  positive  and 
cumulative  effect  upon  the  remaining  developmental  phases. 

Given  that  the  human  factors  engineer  does  understand  the  system 
acquisition  process  and  touches  base  with  the  appropriate 
participants,  he  needs  evidence  of  his  past  and  potential 
contributions.  Although  the  principal  product  represents  a 
process,  this  process  should  yield  a  "product,”  that  product 
being  tangible  documentation  of  the  HF  contribution. 

It  has  been  stressed  that  an  HF  principal  product  is  (or 
should  be)  one  of  the  key  documents  at  each  milestone  in  the 
review  sequence.  Thus,  in  a  broad  sense,  the  principal  product 
has  some  of  the  features  of  a  progress  report,  some  of  the 
features  of  an  historical  record,  and  some  of  the  features  of  a 
promotional  presentation.  Insofar  as  each  milestone  review  can 
determine  the  fate  of  the  system/program,  the  HF  principal 
product  can  be  analogous  to  a  lawyer's  brief — a  kind  of  synoptic 
argument  in  favor  of  a  continuation  of  the  development  activity. 


In  this  context,  the  generic  function  of  the  HF  principal 
product  can  be  denoted  as  follows: 

e  Support  for  the  viability  of  the  system/program. 

e  Support  for  the  specific  design  decisions. 

•  Support  for  the  continuation  of  the  HF  role  within  the 

total  program. 

These  functions  are  highly  interrelated.  As  we  have  seen  in 
the  preceding  discussion,  the  continuation  of  the  investment  in 
an  HF  presence  (staff)  can  be  contingent  upon  the  demonstration 
that  the  design  decisions  recommended  by  the  HF  staff  have  a 
positive  payoff  in  terms  of  reduced  costs  or  enhanced  effectiveness, 
or  both.  Thus,  if  the  design  decisions  can  be  justified  objectively 


and  unequivocally,  the  justification  of  the  HF  role  is  virtually 
automatic.  Similarly,  if  the  design  decisions  are  demonstrably 
constructive,  the  viability  of  the  total  program  is  automatically 
enhanced. 

Implications  for  the  Conduct  of  Human  Factors  RfcD 

At  present  the  HF  role  in  systems  development  typically  is 
limited  to  research  done  in  response  to  basic  human  engineering 
questions  and  to  the  technical  application  of  HF  skills  to 
specific  design  considerations.  While  these  contributions  are 
important,  systems  developers  often  perceive  them  as  simply  a 
means  of  tidying  up  a  system  after  its  configuration  and  basic 
design  are  already  fixed.  HF  staffs  should  not  wait  to  be 
consulted;  they  have  to  be  action-oriented,  "selling  themselves" 
as  real  contributers. 

The  most  difficult  problem  HF  personnel  have  is  that  of 
inserting  themselves  into  the  development  process  in  the  Mission 
Analysis  phase  before  a  system  is  officially  approved  by  the 
dod.  Although  DOD  documentation  requires  human  factors  participation 
in  all  phases  of  the  acquisition  cycle,  systems  initiatives  may 
begin  in  any  number  of  organizations,  and  there  exists  no 
machinery  to  ensure  early  human  factors  participation.  If  HF 
staffs  are  to  influence  the  direction  of  system  concepts  such 
that  human  performance  considerations  will  be  built  into  basic 
system  configuration  and  design,  their  ability  to  make  real 
contributions  must  be  known.  A  rapport  between  HF  people  and 
systems  developers,  combat  developers,  etc.,  should  be  cultivated 
so  that  those  who  initiate  systems  will  seek  help  from  the  HF 
community  at  the  earliest  possible  moment.  The  human  factors 
practitioner  can  be  invaluable  in  presenting  "lessons  learned” 
from  predecessor  systems,  applicable  research  findings  which 
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bear  upon  the  system  concept,  and  the  known  impacts  of  alternative 
design  options.  These  considerations  can  then  can  be  incorporated 
into  official  documents,  such  as  the  ROC  and  MENS. 

Once  the  MENS  is  approved,  two  significant  events  occur  in 
sequence:  (a) the  PM  is  selected,  and  (b)  the  prime  contractor 
is  chosen.  Whether  or  not  the  HF  practitioner  has  a  role  in 
this  selection  process,  his  nontechnical  function  should  now  be 
that  of  directly  supporting  the  PM.  He  should  make  clear  the 
value  of  various  functional  and  design  alternatives,  assuring 
the  PM  that  good  human  factors  not  only  will  provide  a  form  of 
insurance  against  major  failure  but  also  will  increase  the 
chances  of  a  highly  successful  system.  He  should  also  monitor 
as  closely  the  HF  performance  of  the  prime  contractor  and 
interpret  human  factors  data,  whatever  its  source,  to  the  PM  in 
order  that  it  can  be  acted  upon  intelligently.  Finally,  the 
HF  practitioner  should  help  to  interpret  the  system  to  those 
who  are  outside  the  program  staff  but  who  can  control,  in  one 
way  or  another,  the  fate  of  the  system.  In  particular,  this 
function  requires  the  HF  staff  member  to  make  a  special  contri¬ 
bution  to  the  milestone  reviews  (e.g.,  DSARCs)  and,  in  some 
instances,  to  actually  present  "evidence”  at  such  reviews. 

’ Recommendations 

There  are  three  substantive  actions  which  would  vastly 
improve  the  current  state  of  the  art  in  HF  utilization,  analysis, 
and  evaluation  in  military  system  development.  These  actions 
are  based  in  part  on  the  case  study  experiences,  and  in  part  on 
the  overall  study  results. 

1.  Handbooks  are  required  that  contain: 

e  Guidance  for  the  preparation  of  HF  principal  products 

and  the  application  of  the  impact  assessment  methodology. 
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•  Data  and  reference  information  on  cost  and  planning 
factors  for  personnel  and  selected  reference  systems 
that  are  pertinent  to  HF  analyses.  Also,  an  impact 
metric  vocabulary  would  be  included. 

e  Models  and  their  documentation  for  use  in  impact  assessments. 
Selection  criteria  and  application  references  should 
also  be  included. 

e  Case  Examples  of  both  the  HF  principal  products  and 

impact  assessments  with  references  for  follow-up  inquiries 
to  the  particular  practitioners. 

Two  handbooks  are  recommended:  one  that  focuses  on  the 
system  developer  and  management  community  task  of  defining 
and  reviewing  the  context  of  HF  principal  products  and 
impact  assessments;  and  a  second  that  focuses  on  the  analyst 
who  prepares  the  HF  principal  product  and  impact  assessment. 

Both  handbooks  would  have  a  common  goal  of  ensuring  that  the 
HF  inputs  would  be  relevant  to  and  incorporated  into  the 
system  development  process. 

The  handbooks  should  be  designed  in  such  a  way  as  to  permit 
easy  updates  and  additions  cn  a  periodic  basis. 

2.  A  Policy  of  Aggressive  Participation  on  the  part  of 
human  factors  centers  of  expertise  to  support  instruc¬ 
tion  programs,  and  actual  preparation  of  HF  principal 
products  and  impact  assessments.  The  latter  should  take 
place  where  the  major  system  project  offices  are  located. 

3.  An  Information  System  to  collect,  store,  and  retrieve  HF 
principal  products  and  impact  assessments  and  their 
input  data.  This  information  system  should  represent 


all  important  or  HF-aensitive  functions  in  the  military 
and  permit  flexible  retrieval  of  HF  study  data  in  a 
timely  and  convenient  manner. 

Summary 

The  primary  supposition  of  this  project  has  been  that  human 
factors  engineers  need  to  objectively  demonstrate  their  accomplish 
ments  in  the  development  of  military  systems.  This  is  necessary 
if  system  developers  are  to  perceive  HF  as  a  discipline  which 
offers  a  substantial  contribution.  The  Phase  I  report  presented 
the  rationale  for  the  inclusion  of  HF  in  systems  development. 

The  main  elements  of  that  report  were  as  follows:  examples  of  HF 
deficiencies/contributions;  an  analysis  of  the  acquisition  cycle; 
and  the  concept  of  human  factors  in  systems  development. 

The  objective  of  the  present  (Phase  II)  effort  has  been  to 
demonstrate  the  human  factors  principal  product  concept  and  the 
impact  analysis  framework  (PPIAF)  approach  by  the  use  of  case 
studies.  This  proved  to  be  an  illuminating  endeavor.  An  attempt 
to  retrospectively  assemble  a  principal  product  led  to  a  number 
of  conclusions:  (1)  HF  efforts  require  more  careful  documentation 
(2)  Mission  Analysis  principal  products  are  the  most  crucial  but 
least  likely  to  be  developed;  (3)  since  system  development  often 
does  not  directly  follow  the  designated  DOD  guidelines,  the 
principal  products  of  reference  systems  are  an  important  consider- 
ation;  (4)  an  organizational  system- by-HF  issue  taxonomy  and 
related  data  b*nk  are  needed  if  the  HF  community  is  to  take  a 
more  active  role  in  gaining  influence  among  the  organizations  and 
personnel  who  dictate  the  course  of  system  development.  In  regard 
to  the  demonstration  of  impact  analysis  as  a  tool  for  measuring 
the  contributions  of  HF,  both  disadvantages  and  advantages  of  the 


method  emerged.  Data  are  often  inadequate ,  and  there  are 
language  gaps  between  engineers  and  HF  practitioners.  However 
impact  analysis  provides  both  a  framework  for  comparing  and 
selecting  HF  alternatives  and  a  discipline  for  identifying 
relevant  human  factors  issues.  In  addition,  it  permits  human 
factors  to  be  represented  as  a  parameter  in  the  systems 
acquisition  cycle. 
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APPENDIX  A 

CASE  STUDY  INVESTIGATIONS  OF 
PRINCIPAL  PRODUCTS  AND  IMPACT  ANALYSES 


The  case  study  approach  both  provides  a  base  for  the  impact 
analyses  and  conveys  a  sense  of  what  actual  principal  products 
will  look  like  when  assembled  during  the  course  of  system 
development.  The  approach  allows  researchers  to  insert  them¬ 
selves  into  a  position  resembling  that  of  the  HF  staff.  This 
in  turn  provides  a  realistic  perspective  from  which  to  view  the 
problems  of  tracking  the  human  factors  effort  to  assemble  a 
principal  product.  A  case  study  also  makes  available  specific 
information  about  costs,  deficiencies,  and  fixes.  This  infor¬ 
mation  can  be  weighed  to  make  choices  among  various  system 
characteristics  that  will  be  appropriate  in  the  demonstration 
of  impact  analysis.  In  addition,  the  implementation  of  the 
methodology  within  the  framework  of  an  actual  system  enhances 
credibility.  Finally,  the  case  study  approach  helps  to  elucidate 
the  problems  of  implementing  a  precise  methodology  in  a  situation 
in  which  human  factors  impacts  have  usually  been  evaluated 
subjectively. 

A  few  additional  comments  are  appropriate  at  this  point.  In 
attempting  to  apply  this  methodology  to  systems  that  were  already 
well  into  development,  several  problems  became  apparent.  The 
most  significant  problem  was  the  fact  that  the  case  study  systems 
had  not  been  developed  according  to  the  approach  described  in 
the  Phase  I  report.  This  meant  that  human  factors  engineering 
personnel  were  not  necessarily  able  to  participate  as  suggested 
by  our  approach;  and,  it  also  meant  that  the  data  required  for 
developing  the  principal  products  were  not  necessarily  generated. 
Consequently,  the  principal  products  could  not  be  created  for  the 
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case  studies  according  to  the  requirements  specified  in  the 
Phase  I  report.  Limited  availability  of  data  was  a  principal 
constraint  on  the  impact  analysis,  necessitating  certain 
assumptions  when  data  gaps  existed. 

The  Case  Studies 


Prior  to  the  case  presentations,  a  few  comments  about 
the  case  descriptions,  derivation  of  principal  products,  and 
methodology  are  appropriate. 

Case  Descriptions 

The  case  system  selected  are  the  F/A-18  (Navy)  and  SIGMA 
(Army) .  Because  SIGMA  (a  maneuver  control  system)  is  still  in 
a  very  early  stage  of  development,  the  HF  issue  chosen  for 
demonstrating  the  methodology  is  relevant  to  the  Milestone  0 
decision  point.  Thus,  the  system  described  is  actually  a  generic 
computerized  system  for  maneuver  control.  That  is,  the  Mission 
Analysis  questions  pertaining  to  threat,  reference  system,  role 
of  man,  etc. ,  essentially  apply  to  any  automated  maneuver  control 
system  at  this  point  in  time. 

Principal  Products 

In  the  Maneuver  Control  case  study,  the  principal  product 
was  patterned  after  the  Role-of-Man  Statement  described  in 
Chapter  2.  This  was  appropriate  because  the  Milestone  0  phase 
is  being  addressed  in  that  case  study.  In  the  F/A-18  case  study, 
the  methodology  is  applied  to  HF  actions,  decisions,  and  issues 
arising  during  the  Milestone.il  phase;  the  principal  product 
embodies  the  Task  Analysis  and  Human  Engineering  Requirements 
discussed  in  Chapter  2.  In  neither  case  should  the  principal 
product  be  considered  complete.  It  has  become  clear  during  the 


course  of  the  project  that  the  derivation  of  a  principal  product 
by  an  HF  staff  during  system  development  is  a  major  job;  doing  so 
retrospectively  is  infinitely  more  difficult.  Stated  differently, 
the  basic  problem  addressed  by  this  report  has  become  manifest  in 
the  derivation  of  principal  products  from  available  documentation. 
Thus,  the  products  illustrated  in  the  report  are  limited  examples 
of  those  which  a  systems  HF  staff  should  be  consolidating  and 
documenting  as  development  progresses,  and  the  content  of  the 
products  does  not  match  the  ideal  requirements  delineated  in 
the  Phase  I  report.  Finally,  the  choice  of  Milestones  0  and  II 
products  reflects  an  increasing  awareness  of  the  need  for  early 
HF  inputs  in  systems  development.  From  a  human  engineering 
standpoint,  the  human  performance  failures  in  a  system  usually 
can  be  traced  to  the  lack  of  HF  considerations  in  the  early 
conceptual  stages  of  a  system. 


The  impact  analysis  for  each  case  is  presented  next.  The 
methodology  is  rigorous,  but  in  cases  where  there  are  gaps  in  the 
data  because  of  incomplete  HF  information,  certain  assumptions 
had  to  be  made,  This  is  appropriate  since  the  object  of  this 
section  is  the  demonstration  of  the  ’"'’thodology;  the  limited 
availability  of  some  data  makes  the  demonstration  no  less  valid. 


Case  Study  1:  The  F/A-18  HORNET 


General  System  Description 


The  F/A-18  "HORNET"  is  a  twin  engine,  all-weather,  light 
attack  fighter  that  is  intended  to  replace  the  A-7,  A-4,  and 
F-4  currently  being  used  by  the  Navy  and  Marines.  The  F/A-18 
Project  Manager's  office  is  located  at  the  Naval  Air  Systems 
Command.  McDonnell-Dduglas  is  the  prime  contractor,  and  three 
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other  major  contractors  are  involved  in  the  effort:  Northrop 
(primary  fuselage) ;  Hughes  (radar) ;  and  GE  (engines) .  The 
aircraft,  which  went  into  full-scale  development  in  1976,  was 
first  flown  in  late  1978  and  has  logged  over  300  flight  hours. 

Nine  pilot-production  HORNETS  were  built  from  FY  1979  funding 
appropriations,  and  an  additional  25  limited  production  aircraft 
were  ordered  for  FY  1980. 

The  HORNET  is  built  to  deliver  17,000  pounds  of  ordance; 
its  potential  range  should  exceed  500  miles  on  an  attack  mission. 
It  is  expected  to  have  a  combat  ceiling  of  about  50,000  feet  and 
maximum  speed  of  over  1.8  Mach.  The  HORNET'S  armament  includes  a 
20mm  gun  and  the  Sparrow,  Sidewinder,  and  Harm  missiles.  It  can 
also  carry  other  missiles  and  bombs,  including  nuclear  weapons. 

The  F/A-18  is  quite  advanced  in  both  operational  and 
maintenance  capability.  Controls  and  displays  are  designed 
to  facilitate  rapid  response  by  the  pilot  in  high-speed  combat 
situations.  The  cockpit  is  fully  automated  and  reflects  the 
"HOTAS"  concept  (hands  on  throttle  and  stick) .  Four  cathode 
ray  tubes  have  replaced  the  familiar  maze  of  dials.  Via  the 
throttle,  stick,  and  up-front  control  panel  the  pilot  can  obtain 
needed  information  by  requesting  the  MENU  and  keying  in  the 
request.  The  information  about  navigation,  target  tracking  and 
acquisition,  stores  inventory,  etc.,  is  displayed  on  the  HUD 
(head-up  display)  or  on  one  of  the  four  cathode  ray  tubes  present 
in  the  cockpit,  obviating  the  need  for  the  pilot  to  divert  his 
eyes  from  the  target.  The  radar  provides  excellent  target 
definition  at  long  range  and  allows  for  the  scanning  of  multiple 
targets  while  locked  onto  others. 

The  combination  of  high  speed  (both  F/A-18  and  target) ,  the 
requirement  for  instantaneous  decision  in  combat,  and  the  transfer 


from  air-to-air  (a/a)  combat  to  air-to-ground  (a/g)  attack  led 
to  a  hardware/software  design  which  has  a  substantial  analysis 
capacity  and  greatly  reduces  the  pilot's  cognitive  load.  Many 
of  the  more  critical  navigational  and  weapon  system  controls  and 
displays  are  located  together  to  produce  little  break  between 
strictly  navigational  and  combat  activities.  For  example, 
displayed  information  about  weapon  stores,  etc.  may  be  overridden 
by  airspeed  or  angle  of  attack  warnings.  The  main  displays  (see 
Exhibit  A-l)  contain  redundant  information  in  order  to  reduce  the 
monitoring  load  that  results  from  visual  shifts  between  displays. 
In  addition,  the  displays  incorporate  various  declutter  and 
boxing  options. 

The  main  control  functions  of  the  cockpit  are  broken  down 
into  three  "modes”  (navigational,  a/a,  and  a/g) ,  and  the  mode 
selected  determines  which  weapons  are  activated,  the  parameters 
displayed,  the  appropriate  displays,  and  the  slewing  of  the 
sensors.  The  driving  force  is  the  HOTAS  concept,  referred  to 
earlier.  Response  capabilities  were  built  into  the  stick  and 
throttle  to  allow  the  pilot  to  implement  decisions  instanta¬ 
neously  without  removing  his  hands  from  either  control;  this 
also  obviates  the  interruption  of  basic  navigational  procedures 
in  combat.  Also,  various  controls  are  built  into  vhe  instrument 
panel  to  serve  functions  which  are  not  usually  combat  related 
(e.g.,  signal  analysis)  and,  in  some  cases,  to  allow  alternatives 
to  throttle  or  stick  control  of  combat  modes. 

The  F/A-18  system  was  designed  to  greatly  reduce  the  cost 
and  time  of  maintenance.  Much  of  the  technology  is  modular 
(e.g.,  radar,  engines);  the  power  plant  of  the  F/A-18  conse¬ 
quently  has  over  7,000  fewer  parts  than  that  of  the  F-4.  Not 
only  are  engine  changes  easier,  but  most  of  the  parts  for  the 
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right  and  left  engines  are  interchangeable .  The  Navy  is 
developing  the  electronic  test  systems  required  for  the  F/A-] 8 
avionics  and  radar.  These  are  called  the  Intermediate  Level 
Avionics  Support  System  (ILASS)  and  the  Radar  Test  Station 
(RTS).  Originally  these  were  conceived  as  single-port  systems, 
but  recent  analyses  suggest  that  development  of  a  dual-port  ILASS 
and  multi-port  RTS  would  reduce  costs  and  increase  efficiency. 

In  order  to  reduce  the  amount  of  depot  maintenance  and 
required  number  of  pipeline  aircraft,  the  Navy  has  begun  imple¬ 
menting  Reliability  Centered  Maintenance  (RCM) .  The  RCM  approach 
entails  both  the  rating  of  components  in  terms  of  criticality 
and  reliability  and  a  reduction  in  the  amount  of  "hard  time" 

(scheduled)  maintenance  by  the  use  of  simpler  "on-condition” 
maintenance  tasks.  Such  tasks  are  employed  to  determine  whether 
or  not  an  item  will  remain  in  satisfactory  condition  until  the 
next  scheduled  inspection.  In  general,  the  combination  of 
innovative  design  and  RCM  is  expected  to  reduce  the  number  of 
component  failures  per  hour  of  flying  time  and  decrease  the 
required  number  of  maintenance  hours  and  personnel.  Finally, 
the  implementation  of  a  phased  maintenance  support  program  will 
give  Navy  maintenance  personnel  additional  time  to  learn  the 
system. 

Principal  Product  From  the 
bemons  traiion/Val  lclatlon~PhaBe 

The  following  example  of  the  principal  product  for  the 
F/A-]8  is  intended  to  suggest  the  kind  of  format  that  might 
be  employed  to  illustrate  the  HF  process  during  the  Test  and 
Validation  phase.  The  format  represents  a  flexible  guideline 
which  can  (and  should)  be  tailored  to  the  actual  development 
process  of  a  given  system.  The  steps  are  not, invariant  in  their 

:  1 

:  1 
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order,  and  they  should  not  be  considered  all-inclusive.  If  the 
approach  is  used  iteratively,  it  should  help  to  provide  a 
coherent  picture  of  what  the  HF  process  has  accomplished  by  the 
end  of  the  developmental  phase.  In  regard  to  its  ultimate  use, 
the  principal  product  should  yield  a  picture  of  the  human  factors 
decisions  were,  when  and  why  they  were  made,  and  who  made  them. 

The  product  will  effectively  track  the  human  factors  effort  and 
leave  a  trail  of  accountability.  Since  a  variety  of  contractors 
and  government  agencies  often  play  a  role  in  the  human  factors 
aspect  of  systems  development,  this  accountability  is  frequently 
absent.  Also,  the  derivation  of  principal  products  for  a  number 
of  systems  could  serve  a  broader  purpose.  HF  profiles  for 
various  systems  should  illustrate  where  things  go  wrong,  what 
distinguishes  a  successful  from  an  unsuccessful  effort,  etc. 

In  the  example  below,  the  system  related  details  were 
abstracted  from  F/A-18  documentation  but  do  not  necessarily 
represent  either  final  HF  decision  actions  or  the  current  status 
of  the  aircraft,  which  is  now  nearing  the  end  of  Full-Scale 
Development.  However,  the  presentation  of  actual  system  information 
should  provide  a  glimpse  of  the  maze  of  detailed  information 
that  must  be  pulled  together  to  construct  a  principal  product. 

Summary  of  Previous  Milestone  Phases.  A  synopsis  of  the 
basic  human  factors  products  from  Milestones  0  and  1  is  necessary 
since  they  will  have  a  cumulative  effect  and  will  impact  the  HF 
activities  in  the  present  phase.  The  products  of  the  two 
earlier  phases  were  as  follows: 

Milestone  0 — Role  of  Man  determined — One-man  operability 
was  decided  upon  for  the  F/a-18.  Because  of  the  multipurpose 
concept  of  the  aircraft,  the  potential^effects  of  a  highly 
compact,  automated  cockpit  upon  habitability  (space) , 


safety  (ejection  clearance) ,  and  operability/skill  levels 
had  to  be  addressed.  Similarly,  the  effects  of  automation 
and  of  two  sophisticated  weapons  subsystems  on  the  skill 
level  requirements  of  the  maintenance  crews  had  to  be 
considered.  It  was  decided  that  the  impacts  of  the  F/A-18 
concept  on  crewstation  and  maintenance  personnel  would 
warrant  special  consideration  in  the  human  factors  design 
effort,  but  that  they  could  be  dealt  with  effectively. 

Milestone  I — Allocation  of  Functions — The  basic  system 
functions  identified  in  the  Mission  Analysis  phase  were 
evaluated  for  criticality,  capability  of  the  operator  to 
perform,  ability  of  automated  equipment  to  perform,  the 
impact  (of  functions)  upon  each  other,  time  constraints, 
and  optional  ways  of  energizing.  Thus,  for  example,  the 
HOTAS  concept  was  devised  so  that  the  time-critical 
transition  to  the  a/a  mode  can  be  accomplished  via  the 
stick  and  throttle.  Gross  motor  movements  and  changes  in 
visual  position  are  eliminated  by  the  spatial  consolidation 
of  such  functions  as  weapon  selection,  navigational  control 
(air  speed,  attitude,  etc.),  and  sensor  slewing.  On  the 
maintenance  side,  some  of  the  test  functions  often  per¬ 
formed  by  personnel  were  allocated  to  electrical  test 
systems. 

Record  of  HF  Activities  and  Actions.  Human  factors  activ¬ 
ities  and  related  actions  taken  during  designated  time  periods 
(e.g. ,  every  6  months)  should  be  recorded  in  a  manner  such  that 
the  HF  status  of  the  system  can  be  clearly  defined  at  any  given 
time  with  a  minimum  of  effort.  Those  items  to  be  recorded 
should  include  design  concepts  documented  and/or  implemented; 
evaluations  conducted;  analyses  performed;  findings  (problems 
encountered,  issues  raised,  deficiencies  found) ;  and  decisions 
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made.  The  record  should  not  be  a  reetatement  of  the  human 
engineering  plan  or  test  plan,  but  a  summary  of  those  activities 
and  actions  that  have  had  impact  (real  or  potential)  upon  the 
operation  and  maintenance  of  the  system.  Recording  the  HF 
process  in  this  way  should  yield  a  record  that  can  be  readily 
updated  to  illustrate  a  cumulative  human  factors  product  which 
is  comprehensible  even  to  those  having  an  extremely  limited 
knowledge  of  human  factors  engineering. 

The  following  example  is  comprised  of  activities/actions 
occurring  during  a  time  "slice"  of  the  F/A-18  Test  and  Validation 
phase.  Although  the  events  are  real,  their  relationship  in  time 
is  to  some  degree  hypothetical.  The  object  here  is  to  present  a 
tentative  organizational  scheme  rather  than  a  totally  accurate 
description  of  the  system. 

Period  Covered — June,  _  to  Dec. ,  _ . 

A.  Crew  Station 

A-l.  Design/Document  Reviews— The  McDonnell-Douglas 
Human  Engineering  Crew  Station  Design  Document 
illustrates  the  HF  aspects  of  the  basic  cockpit 
geometry  (including  escape  system) ,  ingress/egress 
provisions,  anthropometric  considerations,  and 
control/display  layout  and  rationale.  Comments  on 
this  document  are  included  in  a  letter  from  NAVAIR 
(Commander)  to  McDonnell-Douglas  dated  _ . 

The  following  changes  in  the  document  were  requested 

More  information  about  aural  tones. 

Description  of  lighting  control  sensitivities. 

Much  clearer  description  of  signal  analysis 
picture. 


More  precise  rationale  for  deviations  from  con¬ 
ventional  control/display  philosophy. 

A-2.  Task  Analysis — The  McDonell-Douglas  F-18  Human 

Engineering  Task  Analysis-Part  II  is  primarily  a 
timeline  and  workload  analysis.  The  Task  Analysis 

Report  Review  conducted  by  NADC  on  _ 

included  the  following  comments: 

The  concept  of  one-mem  operability  was  not  fully 
demonstrated  due  to  the  undemanding  requirements 
of  the  simulated  mission. 

The  techniques  used  to  obtain  a  measure  of 
workload  were  not  clearly  demonstrated  or 
explained. 

The  point  at  which  the  pilot  approaches  or 
reaches  overload  is  not  clear. 


A- 3.  Evaluation  of  Escape  System-Simulation — NADC  Tower 

Tests  showed  contact  of  the  foot  with  the  instrument 
panel  during  escape.  Further  tests  (dynamic  foot 
strike,  film  analysis,  instrumented  foot  panel, 
biodynamic  model  computer,  and  test  sled)  are 
anticipated  in  the  near  future. 

A-4.  Air,  Station  Advisory  Panel  (ASAP) —Review  of  Naval 

Weapons  Center  Technical  Memorandum  _ . 

The  panel  generally  agreed  with  the  deficiencies 
and  fixes  recommended  in  the  memorandum  with  a  few 
exceptions.  This  information  is  summarized  and 
presented  in  an  appendix. 
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Maintenance 


B-l.  Design  Documents/Reviews— Critique  of  the  Me  Donne  11- 
Douglas  F-18  Maintainability /Accessibility  Design 
Document  by  the  Commander,  Pacific  Missile  Test 
Center,  includes  the  following  criticisms: 

There  were  few  informative  remarks  concerning 
accessibility,  design,  or  potential  problem  areas 

The  document  does  not  meet  the  requirements  of 
DID  DI-H-2108  since  only  remove  and  replace  tasks 
were  covered. 

‘  Maintainability  of  the  F/A-18  by  personnel  while 
wearing  foul  weather  clothing  was  not  addressed. 

Yellow  Sheet  deficiencies  were  not  addressed  in 
the  document. 

B-2.  Aft/Center  Fuselage  Fixture  Review  (Northrop) — 

(Source — Memo  from  _  to  _ . 

Problems  encountered  were  as  follows: 

APU  Hand  Pump  requires  excessive  time  and  force 
to  pressurize  accumulator. 

AMAD  major  parts  cannot  be  removed  without 
removing  other  components . 

Environmental  Control  System  is  under  standard 
for  accessibility,  clearances,  and  values  which 
operate  in  reverse  of  normal,  etc. 


Action  Items 

e  The  tests  of  the  ejection  system  by  NACD  point  to  a 

greater  than  allowable  probability  of  foot/panel  contact 


Further  testing  and  simulation  should  be  conducted. 

Also,  alternative  designs  and  safety  measures  should  be 
evaluated. 

•  Numerous  inconsistencies  in  control/display  logic  observed 
by  the  ASAP  need  to  be  examined. 

•  The  force  required  to  operate  the  APU  hand  pump  needs  to 
be  examined  further. 

•  Maintenance  time  for  the  VSCV  electrical  power  generator 
is  excessive  due  to  limited  accessibility  to  parts  within 
the  AMAD  Bay.  Design  alternatives  related  to  the  gearbox 
and  fuel  lines  should  be  examined. 

Important  Issues.  Above  and  beyond  specific  deficiencies 
found,  certain  issues  are  of  primary  importance: 

•  One-man  operability  still  needs  to  be  tested  in  a  more 
demanding  environment. 

e  Testing  of  maintenance  tasks  under  adverse  conditions 
has  yet  to  be  done. 

e  Aircrew  task  analyses  have  not  yet  clearly  demonstrated 
aircrew  workload  or  overload  limits. 

e  Both  the  crew  station  and  maintainability  design  documents 
are  at  times  vague  or  lack  sufficient  detail. 


The  above  example  is  intended  to  be  just  that — am  example. 
However,  its  purpose  should  be  re-emphasized.  A  concise,  clear 
method  of  recording  and  updating  the  HF  activities  and  status  of 
a  system  is  necessary  if  the  information  necessary  for  illustrating 
the  costs  and  benefits  of  human  factors  decisions  is  to  be 
obtainable. 

The  following  example  is  a  demonstration  of  how  cost  benefits 
of  HF  alternatives  can  be  derived,  given  that  the  information  is 
indeed  available.  In  the  example  a  few  assumptions  had  to  be 
made,  since  some  of  the  data  necessary  for  such  analysis  were  not 
available. 

Application  of  the  Impact  Assessment 
Methodology — F-l8  Aircrew  Escape  System 

Setting:  This  analysis  is  depicted  as  occurring  at  the 
Milestone  II  point — where  the  experimental  prototype  and  the 
pre-production  prototype  designs  are  reviewed.  The  discussion 
and  analysis  will  necessarily  include  approximations  and  adjust¬ 
ments  for  missing  data  or  macrointerpretations. 

Step  1:  Problem  Definition  and  Solution  Criteria 

Problem.  Analysis  of  the  F-18  escape  system  indicates  a 
high  probability  of  foot  contact  with  the  instrument  panel 
during  pilot  ejection,  and  thus  introduces  the  possibility  of 
pilot  injury. 

Solution  Criteria.  The  general  goal  is  that  any  foot  or 
skin  impact  be  eliminated.  A  less  stringent  goal  is  to  achieve 
the  same  probability  of  foot  contact  as  for  the  F-18  reference 
systems  (A-7E  and  F-4)  . 
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The  objective  of  this  analysis  is  to  identify  the  least 
costly  way  to  achieve  the  above  goals ,  and  to  demonstrate  the 
linkage  between  an  HF  action  and  a  system  design  impact. 

Step  2:  Alternative  Solutions 

The  following  five  alternative  solutions  are  possible  ways 
to  deal  with  the  foot-contact  problem: 

Aq  -  The  Baseline  Design  (Status  Quo  or  "Do-Nothing"  Option) 

A^  -  Change  the  Crew  Station  Geometry  (raise  the  instrument 
panel,  lower  the  heelrest  line) 

A2  -  Crushable  Energy  Absorber 

A^  -  Hinged  Kick  Panel 

A4  -  Passive  Toe  Guide. 

Step  3 :  Define  the  Baseline 

The  F/A-18  strike  fighter  is  a  twin-engine  aircraft  designed 
to  meet  the  Navy's  and  Narine  Corp's  fighter  and  light  attack 
aircraft  requirements.  The  aircraft  is  planned  to  replace  such 
aircraft  as  the  A-7,  A-4,  and  F-4,  now  being  used  for  Navy  and 
Marine  Corps  fighter  and  light  attack  missions  such  as  strike 
escort,  fleet  air  defense,  interdiction,  and  close  air  support. 
The  Navy  also  plans  to  develop  a  reconnaissance  version  of  the 
aircraft  to  replace  the  RF-4  and  RF-8  (GAO,  1981) .  The  baseline 
is  defined  by  the  F/A-18  pre-production  prototype  design  at  the 
DSARC  Milestone  II  point. 

The  crew  station  of  the  pre-production  prototype  F/A-18 
(Exhibit  .A-2)  has  somewhat  less  toe  clearance  than  other  fighter* 
in  service.  (See  the  System  Definition  Statement  below.) 

Design  requirements  for  the  F/A-18  dictated  a  14-inch  toe 


clearance  together  with  a  raised  heelrest  line  rather  than  the 
more  common  16-  to  18-inch  dimensions.  This  geometry,  along 
with  a  rudder  pedal  travel  of  only  ±1  inch  compared  to  ±3-4  inches 
in  other  aircraft,  results  in  the  pilot's  legs  being  in  a 
slightly  straighter  position  than  usual.  This  sitting  position 
has  increased  the  probability  of  foot  contact  with  the  instrument 
panel  during  pilot  ejection. 

4:  System  Definition  Statement  (Milestone  II) 

For  the  purposes  of  this  analysis,  only  portions  of  the 
typical  system  definition  statement  are  needed.  He  have  made 
the  assumption  that  all  data  regarding  Characteristics, 

Acquisition  Policy,  Deployment,  Support  Concept,  and  Logistics 
Goals  data  have  been  reported  in  the  definition  statement  for  the 
F-18  Life  Cycle  Cost  or  Operating  and  Support  Cost  analysis 
documentation  (see,  for  example,  pabbro  &  Fiorello,  1977). 
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Step  5 :  Selection  of 


ict  Areas  and  Metrics 


The  human  factors  empirical  findings  and  observations  can  be 
translated  into  HF  system  engineering  metrics:  crew  accommodations 
and  crew  safety.  Those  metrics  in  turn  can  be  interpreted  as 
having  primary  and  secondary  impacts  on  the  following  system- 
mission  areas: 

Primary  Impact  Area:  Compatibility  with  Aviator  Population 


Secondary  Impacts: 


(a)  Cost  -  Aviator  Training 

Aviator  Recruiting 

Aviator  Retention 

Aviator  Injuries 

Aviator  Assignment  Management 


The  alternatives  identified  in  step  2  will  be  compared  in  terms 
of  their  relative .values  in  the  primary  impact  area  and  their 
respective  implementation  costs. 

Step  6 :  Impact  Assessment  Model 


The  primary  impact  area,  compatibility,  will  be  defined  as 
the  percentage  of  the  Navy  aviator  population  that  could  eject 
from  the  F/A-18  with:  (1)  no  contact,  and  (2)  the  same  expected 
contact  as  for  the  ref erence . systems  (A-7E  and  F-4) . 

The  compatibility  model  is  defined  in  terms  of  a  simple, 
normalised  ratio  of  the  F-18  foot-panel  clearance  relative  to  the 
A-7,  F-4,  and  MIL-STD-1472B  clearances  over  the  standard  deviation 
of  selected  aviator  body  dimension  distributions.  The  ratio, 
denoted  by  A,  is  given  in  equation  form  below: 


Reference  System 
Clearance  . 


Baseline  System 
Clearance 


Standard  Deviation  of  the 
Aviator  Population 


The  ratio  is  in  effect  a  multiple  of  the  standard  deviation 
of  the  aviator  population ,  and  can  be  used  to  define  the  percentage 
of  the  aviator  population  that  the  baseline  system  can  accommodate 
relative  to  the  selected  reference  system.  The  percentile  for 
the  F-18  will  be  defined  relative  to  the  98th  percentile  for  the 
reference  systems.  This  interpretation  permits  a  rough  adjustment 
to  the  typical  aviator  population#  so  that  an  "average”  F>18 
pilot  would  have  the  equivalent  foot-panel  clearance  as  in  the 
reference  systems. 

In  order  to  explore  the  F-18  cockpit  geometry  and  pilot 
foot-panel  clearance  the  following  techniques  will  be  used: 

1.  Mock-up  for  static  tests 

2.  Sled-tower  testing  for  dynamic  measures 

3.  Biodynamic  computer  simulation. 

Important  premises  in  this  formulation  are:  (1)  the  lower 
body  dimensions  are  correlated  with  foot-panel  contact#  and 
(2)  the  A- 7  and  F-4  currently  accomodate  the  98th  percentile 
pilot. 


For  the  secondary  impact  area#  cost#  the  compatibility 
ratio#  A,  could  be  interpreted  qualitatively  into  additional 
costs  for  recruiting#  retention#  administration#  injuries#  and 
management. 

In  this  analysis#  the  alternatives  will  only  be  compared  in 
terms  of  their  impact  on  the  compatibility  ratio  and  selected 
other  decision  criteria#  such  as:  cost  to  implement#  complexity# 
and  weight  added  to  the  aircraft. 

Step  7 :  Collecting  and  Processing  Required  Data 

The  derivation  and  collection  of  the  required  data  include 
the  following  steps: 


Definition  of  the  baseline  design  and  the  reference 
systems  foot-panel  clearance  dimensions.  These  data 
are  shown  in  Exhibit  A-3. 

Interpretation  of  the  aviator  population  in  terms  of 
the  Mans  and  standard  deviations  of  selected  body 
dimensions.  These  data  are  derived  from  MIL-STD-1472B, 
and  are  presented  in  Exhibit  A-4. 

Collection  and  interpretation  of  reference  system  escape 
occurrences  stratified  by  selected  body  dimensions  and 
foot-panel  contact.  These  data  are  presented  in 
Exhibit  A-5,  and  indicate  that  there  is  roughly  a  4:3 
greater  likelihood  of  contact  upon  ejection  for  those 
pilots  who  exceed  the  mean  in  leg  length  and  buttock- 
to-knee  length  than  for  those  who  are  shorter  than  the 
mean. 

Unfortunately,  there  were  no  data  available  on  knee- 
foot  dimensions  and  cockpit  contact  upon  ejection. 

These  data  are  not  conclusive,  but  do  indicate  that 
there  is  a  potential  positive  correlation  between  the 
knee-leg  dimensions  and  panel  contact.  These  obser¬ 
vations  and  findings  by  (Lane,  1971)  and  (Lodge,  1963) 
support  the  hypothesis  that  the  probability  of  contact 
and  injury  is  positively  correlated  with  increasing 
leg  and  buttock- to- knee  lengths.  Other  parameters  are 
also  relevant,  especially  the  dynamic  conditions  at 
the  time  of  ejection,  such  as  air  speed,  sink  rate, 
attitude,  and  center  of  gravity. 

Empirical  testing  and  simulation  analyses  of  the  baseline 
design  and  alternative  solutions  were  also  carried  out. 


Exhibit  A-3 


Foot-Instrument 

Panel  Clearance 

System 

Clearance  (inches) 

F-18 

14 

A-7E 

16.75 

F-4 

18+ 

MIL-STD-1472B 

16 

Exhibit  A-4 

MIL-STD-1472B  Aviator  Population  Dimensions 
Percentile  Values 


Selected  Body 
Dimensions 

5th 

Sitting  height,  erect 

33.7" 

Knee  height,  sitting 

19.3" 

Buttock-knee  length 

22.0" 

Functional  leg  length 

40.9" 

Source:  MIL-STD-1472B 

Standard 


95th 

Kean 

Deviation 

38.8" 

36.3" 

1.55" 

23.6" 

21.5" 

1.3" 

25.8" 

23.4" 

1.16" 

47.4" 

44.15" 

1.98" 

Exhibit  A-S 


Data  on  Ejections:  1969-1977  for 
Navy  Aircraft  (1,2) 


Body  Component 

Contact 

NO 

Contact 

% 

QTY 

% 

QTY 

Leg  length  <  Mean 

50% 

16 

59% 

223 

>  Mean 

50% 

15 

41% 

155 

31 

378 

Buttock-knee  <  Mean 

.  30% 

10 

37% 

138 

>  Mean 

70% 

23 

63% 

235 

33 

373 

Injury 

No  Injury 

% 

QTY 

% 

QTY 

Sitting  height  <  Mean 

38% 

23 

38% 

135 

>  Mean 

62% 

38 

62% 

222 

61 

357 

1.  Data  from:  Shannon,  Ejection  Injuries  from  P.S.  Navy  Aircraft, 
Naval  Air  Station,  June  19^9. 

2.  Means  from:  MIL-STD-1472B  data  for  aviators  and  assumption  of 
a  normal  distribution. 
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A  series  of  tests  was  carried  out  on  the  F-18  escape 
system  including  mock-ups,  computer  modeling,  and  sled- 
tower  dynamic  analysis.  These  tests  confirmed  that 
there  is  a  high  probability  of  cockpit  contact  for 
pilots  when  ejecting  from  the  F-18.  A  Navy  medical 
panel  reviewed  the  test  findings  and  could  not  determine 
conclusively  whether  or  not  contact  would  be  injurious 
to  the  aviator,  and  recommended  that  the  possibility  of 
contact  be  eliminated. 

A  second  series  of  tests  was  conducted  to  gather  data 
on  the  various  alternative  solutions  and  assess  their 
relative  effectiveness  at  reducing  the  probability  of 
contact.  The  data  derived  from  the  tests  are  presented 
in  Exhibit  A- 6. 

Exhibit  A- 6 

Empirical  Test  Data  on  Alternative  Solutions 


Alternative  Solutions 

Foot 

Clearance 

Weight 

Impact 

Implementation 

Complexity 

1. 

Redesign  Crew  Station 

Meet  specs 

High 

High 

2. 

Crushable  Energy  Absorber 

Reduced 

4  lb 

Simple 

3. 

Hinged  Kick  Panel 

Meet*  specs 

5  lb 

Moderate- 

High 

4. 

Passive  Toe  Guide 

Meet*  specs 

<1  lb 

Simple 

♦Appear  to  eliminate  the  potential  of  contact.  The  toe  guide 
provides  the  equivalent  of  increasing  the  foot  clearance  by 
over  4  inches. 


Step  8  s  Setting  the  Conventions 

For  the  primary  impact  area,  the  major  convention  used  is 
that  the  aviator  population  is  normally  distributed  about  the 
mean  value,  and  that  the  pilot  population  for  the  reference 
systems  is  described  by  that  distribution. 


Step  9 :  Estimating  and  Evaluating  the  Impacts 

Based  on  the  discussions  of  the  baseline  design  and  the 
reference  systems,  as  well  as  the  data  analysis,  the  following 
determinations  can  be  made: 


1. 


2. 


The  clearance  ratio.  A,  takes  on  the  following  values  for 
the  reference  systems  indicated  by  the  subscripts: 


"A- 7 


*F-4 


‘1472B 


*  2.1 
=  3 
=  1.5 


Using  the  98th  percentile  (2  standard  deviations  above 
the  mean)  as  the  upper  bound  of  the  compatible  aviator 
population  for  the  A-7  and  F-4,  the  above  clearance 
ratios  equate  to  the  following  percentiles  for  the 
"equivalent"  F-18  aviator  population: 


Relative  to  the  Reference  System 

F/A-18 

Baseline  Design 

AA-7  AF-4 

A1472B 

%  of  F-18 

Aviator 

Population 
with  Equivalent 

46%  16% 

70% 

Compatibility 
to  the  Reference 
Systems 

(Roughly  20  to 

70%  compatible) 

That  is,  tha  bassline  design,  when  compared  to  its  reference 
systems  and  to  MXL-STD-1472B,  would  constrain  the  F-18  aviator 
selection  to  somewhere  between  the  16th  and  70th  percentile. 

These  constraints  would  essentially  make  the  F-18  pilot-cockpit 
contact  occurrence  the  equivalent  of  the  reference  system.  If 
the  possibility  of  all  foot  contact  were  to  be  eliminated,  then 
not  even  drawing  the  F-18  pilots  from  below  the  20th  percentile 
would  be  successful. 

.t 

For  a  dwindling  pilot  retention  rate,  a  dwindling  civilian 
population  to  draw  pilots  from  (estimated  to  be  decreasing  at  the 
rate  of  1  percent  per  year  between  1980  and  1995) ,  and  a  Navy 
combat  pilot  shortage,  the  implications  for  recruiting,  retention, 
and  training  are  all  negative. 


Step  10:  Presenting  the  Results 

The  results  of  the  human  factors  solution  to  the  baseline 
design  compatibility  problem  are  listed  in  Exhibit  A-7.  The 
passive  toe  guide  is  the  most  cost-effective  solution.  Further, 
it  not  only  has  the  potential  to  eliminate  foot  contact  by 
100  percent  in  the  F-18,  but  it  can  also  be  used  on  other  Navy 
aircraft  to  reduce  or  eliminate  foot  contact  in  the  rest  of  the 
fleet. 


Exhibit  A- 7 

Impact  Assessment  Summary 


Impact  Measures 


Alternatives 

Aviator 

Compatibility 

System  Cost 

Performance 
Height  Impact 

*0 

Baseline 

20  to  70* 

Baseline; 

No  change 

None 

Change  Crew 
Station  Geometry 

100% 

High  impact 

NA 

*2 

Crushable  Energy 
Absorber 

• 

Approximately 

10% 

Low 

4  lb 

Hinged  Toe  Guide 

100% 

Medium 

5  lb 

Passive  Toe  Guide 

100% 

Low 

<1  lb 

Cost  to 
Implement 

None 

Very  High 

Low 

Medium 

Low 
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Case  Study  2:  A  Generic  Tactical 
Operating  Maneuver  Control  System 

General  System  Description 

The  Maneuver  Control  System  is  an  automated  network 
which  will  assist  G3/S3  (operations)  in  responding  to  critical 
information  requirements  of  the  commander,  including  the 
extraction  of  information  from  other  functional  area  control 
systems.  Specifically,  it  is  intended  to  facilitate  coordination 
between  Maneuver  Control  and  the  following  centers:  Air  Defense 
Artillery,  Fire  Support,  Intelligence,  and  Combat  Service  Support 
The  system  has  been  proposed  in  response  to  existing  deficiencies 
in  automated  command/control  systems.  The  system  will  be  robust, 
functioning  in  dynamic  environments,  and  will  be  designed  so  as 
to  reduce  information  bottlenecks  at  the  nodes  of  the  system. 

The  primary  piece  of  equipment  is  a  computer  terminal  that, 
along  with  the  associated  software,  will  have  the  following 
capabilities: 

e  Allows  for  the  exchange  of  information  among  all  echelons 

e  Has  a  memory  retention  capability  during  power  loss  or 
fluctuation. 

e  Alerts  operators  of  storage  capacity  approaching  the 
limit. 

e  Facilitates  error  correction  and  has  edit  capability. 

#  Allows  simultaneous  reception/transmission. 

e  Allows  for  the  reconfiguration  of  user  terminals  in  five 
minutes. 

e  Provides  for  off-line  fault  detection  down  to  the  lowest 
replaceable  unit. 

•  Incorporates  camouflage  and  easy  portability. 


The  system  is  conceived  as  a  maneuver  control  for  all 
echelons  from  Corps  down  to  Battalion,  including  the  following 
divisions:  Armor,  Infantry,  Mechanized  (AIM),  Airmobile,  and 
Airborne.  At  the  Corps  and  Division  level  the  system  is  to  be 
located  at  both  the  Main  Command  Post  and  the  Tactical  Operations 
Center,  and  computerized  terminals  are  to  be  placed  in  those  area 
centers  mentioned  earlier  (Air  Defense,  Fire  Support,  etc.). 


Principal  Product  From  the 


Mission  Analysis  Phase 


The  means  chosen  to  address  the  principal  product  was  to 
generate  an  (tentative)  outline  for  a  principal  product  that 
would  be  prepared  during  the  first  phase  of  major  system  devel¬ 
opment  in  the  period  leading  up  to  the  review  process  prior  to 
Milestone  0.  By  and  large,  the  outline  reflected  the  points  in 
the  Phase  I  report,  which  delineates  an  ideal  principal  product 
content.  Three  major  areas  of  concern  are  covered:  operational 
utility,  technical  inputs,  and  management  considerations 
including  both  system  costs  (i.e.,  life  cycle  or  ownership 
costs)  and  development  costs. 

The  first  three  sections  of  the  principal  product  which 
follows  can  be  described  as  brief  background  statements.  The 
main  purpose  is  to  assure  that  the  principal  product  document 
is  a  useful,  intelligible  paper  on  its  own.  While  the  topical 
coverage  of  these  first  three  sections  is  likely  to  be  repe¬ 
titious  with  respect  to  other  program  documents,  they  can  and 
should  convey  a  unique  human  factors  point  of  view. 


The  fourth  section  is  the  heart  of  the  technical  presentation 
and  should  reflect  not  only  the  design/configuration  recommen¬ 
dations  but  also  serve  as  an  archival  record  of  what  the  human 
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factors  contribution  was— in  this  phase— and  how  the  contribution 
was  accomplished  in  a  technical  sense. 

Topic  5,  the  Projected  Development  Plan,  is  also  crucial. 

It  permits  the  HF  representatives  to  respond  to  a  need  on  the 
part  of  program  management  and  at  the  same  time  to  make  the  case 
for  continuity  of  HF  participation. 

Topic  6  is  pro  forma.  The  appendices  would  reflect  the 
need  to  be  comprehensive  in  the  explanation  of  the  methods  used 
to  derive  the  conclusions  asserted  in  the  Topic  4  and  Topic  5 
headings. 

The  Maneuver  Control  System  Milestone  0  principal  product 
which  follows  is  both  incomplete  and,  in  large  part,  hypothetical. 
However,  it  does  serve  as  the  basic  input  to  the  impact  assess¬ 
ment  application,  and  it  is  further  amplified  during  discussion  of 
the  methodology. 


Principal  Product  Statement 
1.  Introduction 
A.  Purpose 

The  purpose  (illustrative)  of  this  report  is  to 
review  (synoptically)  the  development  of  the 
(hypothetical)  system  to  date.  It  identifies  crucial 
issues  in  the  human  factors  area  and  specifies 
deficiencies  that  can  be  used  as  an  analytic  starting 
point  to  justify  continued  participation  by  human 
factors  professionals  in  the  future  development  of 
the  system. 


B.  Logic  of  Approach 

The  format  of  this  report  is  such  as  to  ensure  coverage 
of  three  areas:  Military  operations  (i.e.,  require¬ 
ments)  ;  technical  (technological)  options  and  constraints; 
and  management  aspects  (including  costs  and  scheduling) . 

It  is  intended  to  serve  as  a  source  of  information  for 
high-level  staff  review  of  the  system  development  effort 
and  as  a  source  of  guidance  to  the  Program  Manager  in 
his/her  planning  function  when  such  a  person  is  so 
designated. 

Program  Rationale 

A.  Operational  Problem 

1.  Threat  Environment.  Warsaw  Pact  ground  forces 
outnumber  and  outgun  the  MATO  ground  forces  on  the 
European  continent.  Warsaw  Pact  forces  are  upgrading 
their  own  maneuver  control  capability  and  must  handle 
large*  dispersed  units  in  a  highly  coordinated 
manner 

2.  Military  Objective.  MATO*  and  particularly  U.S.* 
ground  forces  must  be  able  to  redeploy  and  concen¬ 
trate  extremely  rapidly  if  they  are  to  be  able  to 
win  in  the  threat  situation  outlined  above. 

B.  Technological  Opportunity 

Computer  technology*  particularly  microminiaturization 
and  liquid  crystal  display  capabilities*  appears  to 
provide  a  basis  for  the  development  of  compact*  rugged 
systems  for  providing  crucial  information  quickly  to 
tactical  decision-makers. 


C.  Other  Factors 

Present  force  structures  (corporate  level  and  below) 
that  provide  for  independent  mobile  rescues  and  indepen¬ 
dent  strike,  recon,  anti-aircraft,  engineering,  and 
direct  fire  units  are  increasingly  complex,  and  their 
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control  requires  a  multiple,  horizontal  network  for  C  . 

The  resultant  message  traffic  in  both  directions  can 
become  very  heavy. 

Predecessor  System 

A.  Base  Case  Deficiencies 

1.  General.  The  present  system  is  manual  and  hierarchical 
Its  data  storage  (memory)  and  information  retrieval 
capabilities  are  limited.  Adding  more  personnel 

(i.e. ,' headquarters  staff)  contributes  more  to  the 
coordination  load  than  it  serves  to  relieve  the 
information  processing  backlog  in  high  message  traffic 
situations. 

2.  HF-r elated.  Data  from  exercises  indicate  that  the 
delay  from  event  occurrence  to  display  at  corporate 
headquarters  is  8  to  10  hours,  as  against  a  requirement 
for  a  delay  of  less  than  2  hours. 

B.  Upgraded  Base  Case  Option 

The  only  option  for  step-wise  improvement  of  the 
present  manual  system  would  be  some  form  of  partial 
automation.  In  this  case,  that  could  mean  the  creation 
of  computerised  files  for  some  kinds  of  information,  but 
not  for  others.  The  likely  result  would  be  a  "lowest 
common  denominator"  effect  whereby  the  delays  would  be 
driven  by  the  slowest  component  and  the  net  outcome 
would  be  no  improvement. 


New  System  Design  Concept 
A.  Overall  Configuration 


The  basic  concept  is  that  of  an  intelligent  terminal. 
The  essential  functions  include  communication ,  informa¬ 
tion  processing/  storage,  and  display.  The  system  is 
portable  and  rugged.  It  can  be  used  with  minor  peripheral 
alterations  from  the  Battalion  level  on  up. 

B.  Role  of  Man 

1.  Crew/Complement  Composition.  The  system  requires  a 
single  operator  at  the  E-5  to  E-6  level. 

2.  Basic  Assignments  by  Position.  N/A 

3.  Summary  Rationale.  The  configuration  is  essentially 
a  military  adaptation  of  an  advanced  commercial 
version  of  an  intelligent  terminal.  Thus,  while  many 
of  the  hardware  components  are  of  very  recent  vintage, 
they  are  not  state-of-the-art  in  the  usual  sense 
because  they  have  commercial  counterparts  that  are 
off-the-shelf  items.  On  this  basis,  the  initial 
configuration  of  displays  and  controls  (i.e.,  the 
keyboard)  and  the  operating  procedures  are  derived 
from  commercial  practice  and  adapted  to  the  military 
mission. 

Projected  Development  Plan 

A.  Technical  Goals  and  Objectives 

The  immediate  target  is  to  reduce  operator  errors  to 
a  minimum  and  to  speed  up  display  generation  or  regener¬ 
ation,  as  data  uptake  takes  place  in  real  time. 


B.  Problem  to  be  Overcome 


The  prototype  concept  involves  the  use  of  complex 
instructional  codes  (i.e. ,  computer  commands)  and  keying 
procedures  that  are  far  from  self-evident.  This  means 
that  operators  must  be  highly  trained  in  this  specific 
system  and,  in  case  of  battle  casualties,  operator 
replacement  could  be  impossible. 

C.  Approach 

The  approach  suggested  is  to  re-evaluate  both  the 
keyboard  configuration  and  the  procedures  of  use.  It 
is  recognized  that  to  reduce  the  complexity  of  the 
procedures,  the  software  could  become  more  elaborate 
and  voluminous. 

1.  Tasks. 

a.  Man-machine  function  analysis. 

b.  Cost-effectiveness  tradeoff  analysis. 

2.  Staffing.  Six  professional-level  person-months  will 
be  needed  to  arrive  at  a  definitive  design  recom¬ 
mendation  regarding  the  optimum  balance  between  the 
complexity  burden  on  the  human  operator  versus  the 
complexity  of  the  software  plus  reconfiguration  of 
the  prototype  keyboard. 

6.  Summary  and  Conclusions  (outline  only  included  here) 

A.  Need  (for  the  system) 

B.  Conceptual  Response 

C.  Prospects — Implications  of  Next  Developmental  Phases 
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Appendix  1:  Role  of  Man  Analysis 

A.  Mission  Function 

B.  Options 

C.  Procedures  Used  to  Evaluate  Options 
0.  Conclusions. 

Appendix  2:  HF  staffing  Recommendations 

A.  Concept  Development  Phase 

1.  Major  design  issues 

2.  Cost  envelope 

3.  Effectiveness  considerations 

4.  Method  of  integration  (impact  analysis) 

5.  Quantitative  conclusions. 

B.  Demonstration/Validation  Phase 
1.  Approximations 

C.  Full  Scale  Development  Phase 
1.  Approximations 
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Application  of  the  Impact  Assessment  Methodology: 

Tactical  Operating  Maneuver  Control  System 

Setting:  This  analysis  focuses  on  the  human  factors  (HF) -related 
issues  identified  in  the  HF  principal  product  for  the 
Milestone  0  decision.  The  Milestone  0  scope  of  the  HF 
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principal  product  for  C  sy steam  includes  Hole  of  Man, 
Function  Allocation,  and  Task  Definition  considerations 
for  both  operation  and  support  aspects  of  the  mission. 

Step  1 t  Problems,  Goals,  Criteria 

Problems  -  The  HF  principal  product  analysis  has 
identified  the  non-dedicated  user  (operator  and  field  maintainer) 
issue  as  a  significant  HF-related  problem  that  fundamentally 
impacts  the  system  mission  effectiveness  and  life  cycle  support. 

The  problem  can  be  defined  as  the  inability  of  a  person 
not  specially  trained  to  operate  the  terminal  effectively. 

One  could  not  expect  that  even  a  communication  technologist  with 
extensive  experience  in  the  operation  of  conventional  computer 
terminal  devices  would  be  able  to  make  any  sense  out  of  the 
message  codes  and  input  procedures  of  the  particular  device  in 
question.  It  is  known  that  the  exigencies  of  programming  a 
computer  for  any  particular  function— -especially  one  as  compli¬ 
cated  as  maneuver  control — can  lead  to  programing  solutions  that 
"work,"  in  the  sense  that  the  program  runs,  but  that  are  awkward 
and  often  very  complicated  from  the  operator's  point  of  view. 

The  basic  Role-of-Man  issues  are :  How  much  should  the 
machine  do?  and  How  much  should  the  operator  do?  The  problem 
statement  is  how  to  cost-effectively  trade-off  computer  program 
complexity  and  the  complexity  of  the  operator's  job.  This  sort 
of  trade-off  is  at  the  heart  of  the  Role-of-Man  analysis  concept. 
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Goals  -  The  goal  of  the  fielded  system  is  to  achieve 
a  90%  readiness  for  the  primary  mission  during  wartime.  A 
lower-level  goal  is  to  permit  the  non-dedicated  user  to  perform 
essential  send- and- receive  operations  for  the  primary  wartime 
stission  of  the  system.  A  collateral  lower-level  goal  is  to 
achieve  the  above  mission-related  goals  at  an  affordable  life 
cycle  cost. 

Criteria  -  The  achievement  of  the  above  goals  will 
be  determined  by  estimating  the  variable  life  cycle  costs  of 
a  system  design  that  meets  selected  benchmark  tasks  and  non- 
dedicated  user  performance  thresholds. 

Step  2 :  Alternative  Conceptual  Solutions 

For  this  illustration,  only  two  conceptual  alternatives 
to  the  reference  system  will  be  considered. 

Alternative  A  -  An  extreme  case  of  austere,  system- 
specific  computer  programming.  All  the  displayed  messages  are  in 
a  code  and  format  unique  to  the  particular  system,  or  employ  only 
a  small  set  of  "universal"  symbols  such  as  the  map  symbols  used 
by  the  Army  to  designate  categories  of  ground  units.  The  keyboard 
is  in  a  unique,  special  conf iguration  and  the  input  procedures 
are  also  unique.  This  alternative  represents  an  austere  version 
of  the  initial  TOS  tactical  computer  terminal  design. 

Alternative  B  -  The  extreme  opposite  of  Alternative  A, 
in  that  the  computer  programming  is  far  more  elaborate  and 
permits  the  system  to  operate  in  a  virtual  natural  language  mode. 
Some  abbreviations  are  used,  but  both  messages  and  input  proce¬ 
dures  either  are  in  plain  English  or  employ  universal  symbology. 
Furthermore ,  the  system  is  designed  to  be  very  "forgiving,"  in 
thdt,  for  example,  input  errors  are  simply  noted  (on  the  display) 


and  the  operator  is  cued  by  the  machine  about  how  to  rectify  the 
error— as  opposed  to  a  condition  wherein  any  input  error  could 
"jam”  the  program.  This  alternative  represents  an  advanced 
design  with  more  systems  capabilities  than  are  present  on  the 
contemporary  tactical  computer  terminal/system. 

Step  3 :  Define  the  Baseline 

The  baseline  is  the  system  design  that  exists  at  the 
beginning  of  the  analysis.  For  example,  in  the  analysis  during 
the  Milestone  I  phase,  the  baseline  is  the  system  design (s)  that 
resulted  from  the  Milestone  0  analysis.  In  the  case  of  the 
Milestone  0  analysis,  the  baseline  is  often  defined  in  terms  of 
the  existing  mission-reference  system.  Thus,  for  this  analysis, 
the  baseline  is  represented  by  the  present,  manual  Maneuver 
Control  System.  The  typical  operation  of  the  manual  Maneuver 
Control  System  involves  voice  and  teletype  inputs  to  a  map 
plotter  in  the  Tactical  Operations  Center.  Delays  and  errors 
due  to  message  overload  are  notorious  deficiencies  of  the 
existing  system.  Moreover,  the  capability  to  coordinate 
maneuvers  based  on  a  common  representation  of  the  battle 
environment  at  all  levels  of  command  does  not  exist  in  the  manual 
system. 

The  reference  system  does  have  the  advantage  of  being 
able  to  use  operator  personnel  whose  training  need  not  be  system- 
specific. 

Step  4  s  System  Definition  Statement 

3 

For  the  C  force  level  and  maneuver  control  mission 
analysis  of  Milestone  0,  only  the  following  definition  statement 
components  are  required. 
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Mission:  Desi 


[uirements  - 


qn  Reg 

a.  General  -  To  command  and  control  tactical  units, 
including  the  capability  to  provide  survivable, 
recons titu table,  secure,  and  interoperable  means 
for  tactical  force  management  and  technical  support 
of  nuclear  and  general  purpose  force  operations. 

b.  Timely  Processing  of  Information  - 

(1)  Allocation  or  reallocation  of  maneuver  and  fire 
support  units  within  2  hours. 

(2)  Carrying  out  of  conventional  situation  assess¬ 
ment,  decisionmaking,  and  dissemination  of 
orders  within  3  hours. 

(3)  Developing  situation  assessment  products  within 
20  minutes  at  Corps  and  Division. 

(4)  Ensuring  that  all  assessment  information 
(friendly  and  enemy)  is  current  to  within 
1  hour. 

c.  Continuity  of  Combat  Operations  - 

(1)  Reeponeiveneaa .  The  ability  to  rapidly 
disseminate  information  to  all  levels  of 
command.  It  is  characterized  by  the  capability 
for  having  time- sensitive  information  available 
during  the  decisionmaking  process. 

(2)  Survivability /Security .  The  ability  of  the 
system  to  deny  and/or  withstand  enemy  radio 
electronic  combat  (EEC) .  It  is  characterized 
by  the  ability  to  minimize  the  effects  of  enemy 
efforts  to  intercept,  monitor,  analyze,  locate, 
and  target  friendly  forces,  and  by  the  ability 
to  survive  physical  attack.  * 


(3)  Dependability.  The  ability  to  ensure  that 
critical  information  ia  exchanged  among  users 
with  minimum  loss  of  accuracy.  It  is  charac¬ 
terised  by  the  ability  to  provide  both  a  high 
degree  of  reliability,  availability,  and 
maintainability  in  a  highly  mobile  tactical 
environment,  and  efficient  handling  of  traffic 
loads  with  ranges  of  750  to  1100  messages  per 
hour. 

(4)  Flexibility .  The  ability  to  be  rapidly 
deployed  ami  employed  to  support  ground  combat 
by  providing  critical  information.  It  is 
character  lied  by  the  capability  to  provide 
continuous  information  through  various 
communications  means. 

(5)  Interoperability .  The  capability  to  interact 
with  existing  and  programmed  information 
systems  of  ground  combat  used  by  other  services, 
and  with  the  command  and  control  systems  of 
allied  nations. 

Design;  Operational  Characteristics  - 


a.  Display  -  Produce  hard  copy  (alphanumeric/graphic) 
at  the  same  scale  as  display.  Also,  large  screen 
display;  declutter,  re-arrangement  of  symbols,  etc. 

b.  Storage/Retrieval  -  Save  symbols,  memory  retention 
during  power  loss,  alert  user  about  storage  capacity 
approaching  limits . 

c.  Composition  -  Save  distribution  lists  for  multiple 
addresses;  minimal  user  action*  Composition  aided 
through  prompts;  valid  entries  for  fixed  formats 
will  be  pre-defined. 


Reception/Transmission  -  Simultaneous  reception/ 
transmission  without  interference  of  the  message 
preparation;  when  load  over  peak,  graceful 
degradation  by  discontinuing  information  flow  in 
inverse  order  of  priority. 

Edit  Capability/Error  Detection  -  User  can  modify 
message  without  deleting  or  recreating  the  message. 
A  communication  error  detection  capability  also 
will  exist. 

Keyboard/Compatibility  -  Keyboard  designed  so  that 
user  can  work  while  wearing  protective  clothing. 

User  Requirements/Training  -  Designed  to  be  used 
by  those  personnel  meeting  the  requirements  of  the 
user  population  projected  by  TRADOC.  TRADOC  will 
update  the  Individual  and  Collective  Training  Plan 
(ICTD) . 

Design  Life  Expectancy  -  10  years. 

Fault  Isolation  -  Provide  for  on/off-line  fault 
detection  down  to  lowest  replaceable  unit. 

Data  Distribution  Considerations  -  The  system  must 
be  designed  to  operate  in  the  current  communication 
environment  as  well  as  with  emerging  communication 
systems.  There  is  no  intention  to  develop  a 
dedicated  communications  system  to  satisfy  this 
need. 


Acaui sition/Deplo 


Development  Costs 


Procurement  Costs 


I 


Deployment  -  Mid  1980s 


-  European/NATO  setting/Corps , 

.  Division/Battalipn,  Company 

-  Systems  will  be  deployed  as  part  of 
existing  maneuver  control  organization. 

>ort  Considerations  - 

Compatibility  - 

(1)  Improved  capabilities  must  be  supportable  and 
compatible  with  existing  and  future  logistic 
concepts.  Design  configurations  should  be 
appropriate  to  the  employment  environment, 
recognizing  the  requirements  for  system  mobility 
for  ground  maneuver  units,  as  well  as  life 
cycle  costs. 

(2)  The  system  must  be  designed  to  minimize  the 
need  for  high- ski 11  personnel,  and  must  not 
exceed  the  minimum  expected  skill  level 
(prerequisite  aptitude  score)  of  maintenance 
and  operating  personnel  for  generically  similar 
equipment  existing  in  the  field. 

(3)  Operators  will  perform  field  level  (1st  echelon) 
maintenance . 

Training  - 

(1)  A  Training  Subsystem  must  be  developed  to 
provide  for  a  transfer  of  knowledge  to  the 
system  user  and  maintainer.  The  training 
package  must  be  designed  to  be  cost  effective 
within  the  limits  of  training  constraints. 


A  summary  comparing  the  ref,:>rence  system  and  the  two  concep¬ 
tual  alternatives  is  provided  in  Exhibit  A-8.  The  major  difference 
between  the  two  alternatives  is  the  man-machine  distribution  of 
the  workload  burden  and  the  consequent  life  cycle  cost  and  mission 
impacts.  Both  Conceptual  Alternatives  are  expected  to  meet  the 
mission  specifications,  principally  through  the  introduction/ 
utilization  of  off-the-shelf  and  state-of-the-art  technologies. 


Step  5 :  Selection  of  Impact  Areas/Metrics/Empirical  Measures 

The  reference  system  does  not  satisfy  the  current  and 
forecasted  mission  requirements.  The  two  conceptual  alternatives 
are  designed  to  satisfy  the  mission  needs,  but  will  have  different 
cost  and  compatibility  values.  Thus,  for  this  analysis,  the 
formulation  will  be  based  on  a  fixed  effectiveness  threshold  and 
variable  cost  and  compatibility  comparisons.  The  Impact  Areas  of 
interest  are  Cost  and  Compatibility. 

Each  of  the  Impact  Areas  can  be  defined  in  terms  of 
selected  metrics  that  provide  more  specific  breakouts  of  the 
alternative  design  impacts  on  the  current  Army  organizations  and 
budget. 

For  the  Impact  Area  of  Cost,  lower-level  metrics  can  be 
selected  from  the  Joint  Tactical  Communications  Systems/Equipment 
Life  Cycle  Cost  Model  (Report  TTO-ORT-032-76B-V3,  Joint  Tactical 
Communications  Office  (TRI-TAC) ;  and  the  Army  Materiel  Command, 
Pamphlets  11-2,  11-3,  and  11-4) .  For  Milestone  0,  virtually  all 
system  life  cycle  costs  are  variable,  in  that  they  have  not  yet 
been  incurred  and  current  design  decisions  can'  affect  them. 
However,  this  formulation  is  concerned  with  the  comparative 
differences  between  two  alternatives,  and  only  the  relevant 
costs — that  is,  those  that  are  variable  between  the  alternatives— 
are  needed.  The  selected  relevant,  variable  costs  are  identified 
in  Exhibit  A-9. 


Comparison  of  Alternative  System  Concepts 


Exhibit  A-9 

Selected  Cost  and  Compatibility  Metrics 


Impact  Ana 

MtuKI 

Name 

Symbol 

Empirical 

Measure 

Lift  Cycle  Cost 

Research  and  Development 
-  Software  Development  and  Hardware  Design  Fabri¬ 
cation 

Crd 

Dollars 

Investment 
—  Acquistion 

>° 

o 

_ i _ 

Dollars 

Operation  and  Support 
-  Operators 
—  Training 

—  Replenishment  Spares 

Coas 

Dollars 

Dollars 

Dollars 

Compatibility 

Skill  Requirements 

(SR) 

%  of  current  Tactical 
Operation  Systems 
personnel 

Task  Loading 

(TL) 

Ratio  of  relative 
loading 

The  selected  metrics  for  the  second  Impact  Area, 
Compatibility,  are  also  listed  in  Exhibit  A-9 .  The  metrics  for 
measuring  compatibility  include:  Skill  Requirements ,  and 
Functions  and  Task  Loading. 

Step  6 :  Construction  of  the  Impact  Assessment  Model 

For  this  Milestone  0  analysis,  macro  planning  factor 
models  are  appropriate.  The  basic  equations  for  each  of  the  cost 
metrics  are  taken  from  the  Army  Life  Cycle  Cost  Model  and/or  the 
TRI-TAC  Tactical  Communications  Life  Cycle  Model,  and  are  listed 
below. 

Cost  of  Research  and  Development  (CRD* 

C,  *  Cost  of  Research  and  Development 

1.10 

Development  Engineering  Cost 
Producibility  Engineering  and  Planning  cost 
R&D  Tooling  Cost 

Prototype  Manufacturing  Cost  (includes 
software  development  costs) 

R&D  Data  Cost 

R&D  Test  and  Evaluation  Cost 
R&D  System/Project  Management  Cost 
R&D  Training  Services  and  Equipment  Cost 
R&D  Facilities  Cost 


“23 

i*1.01 
where,  C 


Ci 


1.01 

'1.02 

'1.03 


'1.04 


1.05 

1.06 

1.07 

1.08 


'1.09 


C.  1  o 


Other  R&D  Cost 


Cost  of  Investma nt  (C^) 
Cn  ■  Investment  cost 


2.11 

z, 

1*2.01 


where,  C2  -  Non-Recurring  Investment  Cost 

C2  02  "  Production  Cost 

C2  Q3  *  Engineering  Changes  Cost 

C2  Q4  -  System  Test  and  Evaluation  Cost 

C2. 05  "  Co,t 

C2  Q6  -  Production  Phase  System/Project  Mgmt  Cost 
C2  07  *  Operational/Site  Activation  Cost 
C2  08  *  Trainin9  Cost  for  10  yr.  Operations 
C2  09  *  initial  Spares  and  Repair  Parts  Cost 
C2  1Q  -  Transportation  Cost 
C„  , ,  «  other  investment  Cost 


Cost  of  Operating  and  Support  (COfcS) 

C3  -  Cost  of  Operating  and  Support 
3.08 

"  2  ct 

i-3. 01 

where,  C3  oi  *  Military  Personnel  Cost 
C3<02  ■  Spare  Parts  Cost 
C3  Q3  ■  Depot  Maintenance  Cost 
C3>04  ■  Materiel  Modifications  Cost 
c3.05  "  oth*r  Direct  Support  Operations  Cost 
C,  M  ■  Indirect  Support  Operations  Cost 


Many  of  tha  above  cost  factors  will  ba  estimated 
diractly  and  based  on  manufacturers '  or  military  historical  data. 
Zf  tha  factors  are  significant  they  will  ba  coaqputad  by  aquations 
that  use  lower- level  elements.  The  set  of  significant  cost 
factors  ist  Cx  04,  C3  06,  Cx  Q8#  C2  02,  C2  og,  C3  Q1#  and  C3  02. 
Examples  to  illustrate  this  type  of  equation  are  given  below. 


Cost  of  Training 


'2.08 


Training  Cost 

(TMYT)  (NTMY)  +  (TE)  (NTE)  +  (TSP)  (TE)  (NTE)  +  (TP) 


where. 


TMYT 

NTMY 

TE 

NTE 

TSP 

TF 


Cost  per  man-year  of  training 
Number  of  man-years  of  training 
Cost  per  training  equipment  set 
Ninber  of  training  equipment  sets 
Training  equipment  spares  factor 
Training  facilities  cost 


Cost  of  Operator  Personnel 

c3  01  "  Military  Personnel  Cost 

-  (N/OS) ($/OP) (HYr) (QTY) 

where,  N/OS  ■  No.  of  operators  per  system 
%/OP  -  Cost  of  operator  personnel 
HYr  -  Operating  hours/yr. 

QTY  -  Quantity  of  operational  equipment 

Spare  Parts  and  Replenishment  Material 

C3.02  “  Spare  Parts  Cost 

-  (IRCF) (EOC) (QTY) 

where,  IRCF  ■  Inventory  replenishment  cost  factor 
EUC  -  Equipment  unit  cost 
QTY  -  Quantity  of  operational  equipment 


Once  the  costs  for  each  of  the  alternatives  are  computed,  the 
difference  between  the  two  can  be  determined  and  the  preferred 
design  identified. 


For  the  compatibility  metrics,  available  historical 
test  and  operational  data  on  fielded  and  experimental  systems 
will  be  assessed  and  interpreted  for  skill  requirements  and  task 
loading  impacts  of  the  two  conceptual  alternatives. 

Step  7 :  Collecting  and  Processing  Required  Data 

Nominal  values  for  selected,  significant  cost  elements 
and  other  driving  factors  are  listed  in  Exhibit  A-10.  For  those 
variables  not  listed,  standard  USA  or  TRI-TAC  factors  were  assumed 

Step  8 :  Establishing  Conventions  and  Assumptions 

For  the  purposes  of  this  analysis,  the  following 
conventions  will  be  used: 

e  Steady  state  operations. 

e  1980  constant  dollars  are  used,  unadjusted  for 
inflation  or  for  the  time  value  of  money. 

e  Technology  and  training  are  off-the-shelf 
capabilities. 

e  The  maneuver  control  units  will  be  deployed  within 
existing  organizations  that  currently  perform  that 
function. 

Step  9 :  Estimating  and  Evaluating  the  Impacts 

Using  the  above  data  and  equations,  the  differences 
between  Alternatives  A  and  B  for  each  of  the  cost  metrics  are 
shown  in  Exhibit  A-ll. 


Ct.04 

- 

A(A-B)  -  $600,000 

C1.0# 

- 

A  (A  O) -*100,000 

C1.0« 

A(A-B)  -  $100,000 

W” 

$290,000 

$300,000 

‘i.oi 

.TMYT13’ 

S  70.000 

$  60.000 

.NTMY 

0.26/oparator 

0.10/eparator 

.TE 

A  (B  A)  *  $20,000 

.NTE 

IS 

15 

.TSP 

0.20 

0.20 

.TF 

$10,000 

$10,000 

^.Oi 

.w OS 

1/ shift  (3  thWts/system) 

1 /shift  (3  shifts/systam) 

.S/OP 

$18.000* 3 > 

$12,000**1 

.HYL 

2020  hrs. /shift 

2920  hrs/shift 

•QTY 

too 

100 

*0.03 

JRCF 

0.18 

0.15 

.EUPC 

$250,000 

$300,000 

.QTY 

100 

100 

Miscellaneous  Factors: 

Operator  Annual  Tumovar 

Rata 

40% 

40% 

System  Operational  Ufa 

10  yr. 

lOyrs. 

(II  Avarae*  aatimatad  east  over  production  run. 

(31  Includes  mumcw  and  tnkm  pay  and  ellowence  pfcwany  PCteapena  adjusted  for  different  win— 


(31  Acquires  an  Hectrp  Mechanical  CotTurwinlcction-Crypso  Synem  IpacMin. 
(41  C-S/C4  lava*  operator. 

i 
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Pro 

SlJM 

®1^S 

®ios 

Othars 

Sub  Total 

A  A  A  A 

fill 

iiSI 

<660,000 

cac 

P»« 

°3M 

Othar* 

Sub  Total 

AC 

<  5,000,000 
16,860,000 

11,860,000 

Poos 

P»S1 

Pis* 

Othars 

SubTot^ 

18,000,000 
<  800000 

17,200,000 

Total  10  yaar  Ilia  cyda  ooat  diffaranoa 

<*S  28,000,000 

The  expected  li£e  cycle  cost  for  Alternative  B  is 
estimated  to  be  about  $28,000,000  (in  constant  1980  dollars) 
less  than  the  life  cycle  cost  of  Alternative  A.  This  is  the 
A  cost  impact  attributable  to  the  investment  in  human  factors- 
related  design  changes  that  distinguishes  concept  B  from  concept  A. 
The  principal  savings  are  in  people- re la ted  categories:  Concept  B 
requires  less  training  and  can  accommodate  a  lower-skilled  and 
-paid  operator.  The  expected  investment  in  the  human  factors 
analysis  and  design  changes  is  less  than  $1,000,000,  which  yields 
a  return  on  investment  of  28:1. 

In  addition  to  the  .cost  savings,  concept  B  provides  a 
capability  beyond  that  of  concept  A:  It  can  accommodate  a  less 
skilled  (non-dedicated)  user  for  the  essential  maneuver  control 
operations  the  system  is  to  support.  That  pays  off  in  increased 
compatibility  and  also  increases  the  operational  readiness  of  the 
system  during  combat  environments. 


APPENDIX  B 

OUTLINE  AND  BOOK  PLAN: 

HF  HANDBOOK  AND  HF  GUIDEBOOK 


•  B-l: 


•  B-2 : 


An  Outline  and  Book  Plan 
for  "A  Human  Factors 
Handbook  for  System  Developers 

An  Outline  and  Book  Plan 
for  "A  Guidebook  for  Human 
Factors  Participants  in  Major 
Military  System  Development 
Programs" 


APPENDIX  B-l 

AN  OUTLINE  AND  BOOK  PLAN  FOR  A 
HUNAN  FACTORS  HANDBOOK  FOR  SYSTEM  DEVELOPERS 


1.  Introduction 

The  outline  presented  below  has  several  special  features. 
First,  it  is  rendered  in  modified  story-board  format.  Second, 
while  we  recognize  the  tri-service  involvement  in  the  project, 
the  terminology  and  conceptual  model  of  military  system  develop¬ 
ment  have  been  derived  primarily  from  the  practices  of  the  U.S. 
Army.  This  was  done  for  the  sake  of  simplicity  and  convenience, 
and  because  the  preparers  were  more  familiar  with  current  Army 
practices.  It  is  hoped  that  the  review  process  will  reveal  where 
the  wording  or  the  concepts  must  be  changed  to  ensure  that  what 
is  being  said  is  valid  for  all  branches  of  the  Armed  Forces. 

2.  General  Instructions  to  Authors, 

Editors,  and  Illustrators 

The  anticipated  primary  mode  of  use  for  the  Handbook  is  as 
an  on-the-job  source  of  reference.  This  mode  of  use  implies 
several  requirements.  First,  each  unit  of  information  or  section 
of  text  should  be  interpretable  by  itself.  The  user  should  not 
be  expected  to  have  to  read  long  narrative  passages  in  order  to 
understand  the  essence  of  each  particular  guide  to  a  course  of 
action.  Secondly,  the  Handbook  should  contain  an  index  to 
facilitate  subject  look-up. 

In  addition  to  its  use  as  a  reference  source,  however,  it 
will  probably  also  be  used  as  a  general  orientation  tool.  This 
means  that  some  background  and  general  explanatory  text  must  be 
provided. 


To  satisfy  these  two  purposes,  a  subtle  shift  in  style  and 
format  should  be  introduced  about  midway  through  the  Handbook . 
This  shift  might  be  described  as  a  transition  from  the  conceptual 
level  of  discourse  (covering  the  orientation  purpose)  to  the 
technical  level  (covering  the  action-guiding  purpose) . 

The  readership  or  audience  will,  we  hope,  be  composed  of 
professional  military  personnel  at  company-grade  rank  and  above, 
civilian  manager /engineer  people  in  government  service  at  or 
above  the  GS-13  level,  and  contractor  personnel  at  the  level 
of  sub- project  team  leaders  and  above.  Most  audience  members 
can  be  expected  to  have  educational  or  experiential  backgrounds 
equivalent  to  an  undergraduate  degree  in  engineering,  at  a 
minimum.  Vocabulary  and  reading  skills  should  therefore  not 
be  a  major  constraint,  but  authors  and  editors  should  strive 
to  avoid  jargon  that  is  derived  predominantly  from  the  social 
or  behavioral  sciences  or  even  from  human  factors  engineering 
as  a  specialty. 

We  can  expect  that  the  general  attitude  or  predisposition  on 
the  part  of  prospective  readers  will  be  neutral  or  indifferent. 

In  other  words,  for  most  prospective  users,  the  external  incen¬ 
tives  to  read  the  material  will  be  relatively  weak.  This  means 
that  some  extra  care  should  be  given  to  the  "attractiveness” 
of  the  Handbook.  To  this  end  it  is  suggested  that  physical 
specifications  be  slightly  unconventional. 

•  page  stock:  8>s"  x  11” 

•  binding  open  for  discussion 

•  graphic  (line  drawing)  on  cover 

e  covet  stock  distinctive  color,  medium-heavy  stock  with 
rough  (pebble)  finish 


•  selected  passages  emphasized  by  use  of  a  second  ink  or 
bolder/larger  font 

e  sections  separated  by  lightweight  stock  in  a  color  that 
is  coordinated  with  the  cover  color. 

Along  these  same  lines,  the  narrative  style  should  be  technical 
but  relatively  informal.  Reasonable  models  would  be  Science  81; 
the  Smithsonian's  Natural  History;  or  the  popular  MIT  alumni 
periodical.  Technology  Review.  These  periodicals  are  also  apt 
models  for  the  use  of  graphics,  providing  as  they  do  a  mix  of 
straight  technical  with  more  evocative  items. 

The  format  of  the  Handbook,  as  in  the  following  outline, 
should  be  a  modified  story  board.  That  is,  a  diagram  or 
"bulleted"  set  of  summary  statements  presented  on  the  left  page, 
with  the  narrative  explanation  on  the  right  (facing)  page. 
Substantial  white  space  will  be  unavoidable  on  the  left  pages, 
but  liberal  use  of  white  space  should  also  be  a  feature  of  the 
right  pages. 

The  layout,  then,  would  have  the  following  basic  appearance 


Narrativ* 

Tax* 


3.  Topical  Outline 


Introduction 

(Note:  Each  topic  heading  will  be  printed  across  two  facing  pages.) 

•  The  purpose  and  objectives  of  the  Handbook 
e  Who  should  read  the  Handbook 

•  Why  the  Handbook  should  be  read 

(Note:  Additional  front-matter  is  described  under  "4.  Pagination.") 


Section  1 :  Background 

e  What  is  human  factors  engineering 

e  Historic  contribution  to  military  systems  development 
e  Applications  in  nonmilitary  areas 

•  What  the  human  factors  specialist  is  trying  to  accomplish 
e  How  the  human  factors  specialist  does  his  or  her  job 
e  The  research  side  of  human  factors  work 
e  Links  to  other  disciplines  and  engineering  sub-fields 
e  Some  successful  instances  of  human  factors  work 
e  What  can  happen  if  human  factors  are  ignored 
e  Costs  vs.  payoffs 
e  Limitations 


Section  2;  Managing  the  Human  Factors  Resource 

e  Where  human  factors  specialists  come  from:  Recruitment 

e  What  human  factors  specialists  should  know:  Technical 
Content 


e  What  human  factors  specialists  can  know:  Collateral 
Knowledge 


•  What  human  factors  specialists  should  know  how  to  do: 
Technical  Skills 

e  The  manner  in  which  the  human  factors  specialist  contri¬ 
butes. 

Section  3:  Specific  Contributions  of 
the  Human  Factors  Practitioner 

Administrative  aspects  of  the  human  factors  contribution: 

-  Major  steps  and  decision  points  (OMB  Circular  No.  A-109) 

-  Revised  DOD  directives  (5000  Series) 

-  Other  policy  guides 

-  Military  Standards  and  Specifications. 

The  principal  products  of  the  human  factors  specialist's 
work: 

-  Mission  Analysis  Phase  Human  Factors: 

Human  factors  efforts  and  system  development 
activities  during  mission  analysis 

Content  of  the  Role  of  Man  statement 

-  Concept  Development  Phase  Human  Factors: 

Human  factors  efforts  and  system  development 
activities  during  concept  development 

Content  of  the  Allocation  of  Functions  to.  Man 
statement  as  part  of  the  Decision  Coordinating  Paper 

-  Demonstration/Validation  Phase  Human  Factors: 

Human  factors  efforts  and  system  development  activities 
during  demonstration/validation 

Content  of  the  Task  Analysis  and  Human  Engineering 
Requirements  Product 


-  Full-Scale  Development  Phase  Human  Factors: 


Human  Factors  efforts  and  system  development 
activities  during  full-scale  development 

Content  of  the  Optimal  Man-Machine  Interface  Design 

-  Production  and  Deployment  Phase  Human  Factors: 

Human  Factors  efforts  and  system  development 
activities  during  production  and  deployment 

•  Other  potential  contributions: 

-  Relevant  findings  from  basic  research 

-  Statistical  design  for  system  test  and  evaluation. 

Section  4:  The  Evaluation  of  the 
human  Factors  Contribution 

e  General  criteria 

•  Quantitative  analysis:  Benefit-cost  approach  and  impact 
analysis 

•  Projective  evaluation:  Planning  your  investment  strategy 

•  Comparative  evaluation:  Relationship  to  other  sources  of 
contribution 

•  Retrospective  evaluation:  The  value  of  the  human  factors 
cure 

•  Feedback:  Increasing  value  with  use. 

Section  5:  Rules  of  the  Game 

e  Strategic  considerations 
e  Tactical  considerations 
e  Avoidable  penalties 
e  Prises  to  the  winners. 


A.  Case  study  of  a  system  with  a  wide  range  of  human 
factors  design  deficiencies :  The  Hagen  autoaAtic 
propulsion  system 

B.  Hypothetical  case  study  of  the  projective  type:  An 
Army  C3I  system 

C.  Hypothetical  case  study  of  the  retrospective  type:  The 
F/A-18  pilot  ejection  system 

D.  Procedures  for  conducting  an  impact  analysis  study. 

(Note:  These  latter  components  will  be  in  straight  narrative 
format/  augmented  by  illustrations  included  in  the  text.  That 
is/  there  is  a  shift  here  out  of  the  modified  story— board 
format  into  conventional  technical  report  format.  Also,  note 
that  Appendices  -B,  C  ,  &  D  are  included  in  the  present  report  in 
preliminary  versions.) 
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(Note:  The  appendices  should  be  in  two-column  format.  Each  new 
appendix  begins  on  a  right-hand  page,  with  its  designation  and 
title  centered  at  top.  The  total  number  of  pages  required  for 
appendix  material  cannot  be  specified  at  this  time,  but  could 
run  about  30-40,  printed  on  both  sides.) 

(Note:  The  index  should  be  separated  from  the  appendices  by  the 
colored  stock  used  to  demarcate  each  new  section  throughout— i.e. 
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APPENDIX  B-2 

AN  OUTLINE  AND  BOOK  PLAN  FOR 
"A  GUIDEBOOK  FOR  HUMAN  FACTORS  PARTICIPANTS 
IN  MAJOR  MILITARY  SYSTEM  DEVELOPMENT  PROGRAMS" 

1.  Introduction 

The  document  outlined  below  will  be  a  relatively  conventional 
representative  of  its  genre.  There  is  a  fair-sized  family  of 
handbooks  and  guidebooks  in  the  human  factors  area.  There  are 
also  textbooks  on  human  factors  work  that  have  a  substantial 
weight  of  "how-to-do- it"  content.  Comparisons  reveal  that  vari¬ 
ations  in  format  and  style  within  this  extended  genre  are  marginal 

The  main  differences  between  the  present  document  and  the 
others  in  the  field  will  be  in  purpose  and  content.  The  purpose 
of  the  ordinary  human  factors  handbook  is  to  inform  the  reader 
with  respect  to  technical  substance  (e.g. ,  anthropometric  data) 
and  technical  process  (e.g.,  how  to  conduct  a  task  analysis) . 

The  intent  is  to  improve  the  technical  quality  of  the  product  of 
the  human  factors  specialist's  work  (in  his  or  her  role  as  a 
technician) . 

The  present  document,  by  contrast,  is  intended  to  provide 
information  that  will  help  the  human  factors  worker  ensure  that 
his  or  her  product  is  actually  used  in  a  constructive  manner. 

The  content  is  oriented  toward  such  matters  as  the  overall  nature 
of  the  process,  in  which  the  human  factors  work  is  but  one  part; 
the  organizational  setting;  the  mechanisms  by  which  his/her 
participation  in  the  process  is  initiated;  and,  most  particularly, 
how  the  human  factors  contribution  to  the  total  process  can  be 
evaluated  in  a  relatively  rigorous  fashion  either  by  the  human 
factors  specialist  or  by  the  manager (s)  of  the  total  process. 

To  use  a  military  analogy,  most  human  factors  handbooks  do 
the  equivalent  of  telling  a  soldier  what  a  rifle  is,  and  how  to 


load  it,  aim  it,  and  fire  it  accurately  at  a  target.  The  present 
document  is  more  equivalent  to  telling  the  soldier  how  to  survive 
and  win  on  the  field  of  battle. 

2.  General  Instructions  to  Authors,  Editors  and  Illustrators 

The  modes  of  use  of  the  document  being  planned  will  probably 
be  of  two  kinds,  in  roughly  equal  proportions:  as  a  source  of 
general  orientation;  and  as  a  reference  document.  The  situation 
in  which  the  orientation  function  will  predominate  is  one  in 
which  a  relatively  junior- level  specialist  is  about  to  take  up 
his/her  first  position  as  a  member  of  a  system  planning  or  system 
development  team.  Similar  needs  will  be  present  when  a  person 
who  has  worked  primarily  as  a  researcher  or  research  manager  is 
reassigned  to  development  work,  or  when  a  practitioner  returns  to 
development  work  subsequent  to  a  lengthy  tenure  as  a  teacher, 
researcher,  or  administrator. 

The  document  will  serve  as  a  reference  resource  for  those 
in  the  midst  of  development  work,  and  possibly,  to  a  modest  extent 
for  co-workers  from  other,  non- human  factors  technical  back¬ 
grounds. 

Fulfilling  the  orientation  function  will  mean  that  the  first 
sections  or  chapters  in  the  Guidebook  will  need  to  have  the 
properties  of  logical  flow,  continuity,  and  high  readability. 

The  more  specific  technical  materials  in  the  later  sections  of 
the  Guidebook  will  need  to  have  the  properties  of  explicitness, 
fineness  of  detail,  and  comprehensiveness  to  fulfill  the  reference 
function.  Also,  as  in  the  Handbook  (Appendix  B-l) ,  a  good  index 
is  essential  for  meeting  the  reference  function. 

The  audience  for  the  Guidebook  will  consist  predominantly 
of  human  factors  professionals.  Their  educational  backgrounds 


will  be  uniformly  at  the  first  postgraduate  degree  (e.g. ,  M.S.) 
or  beyond  in  the  behavioral  or  biological  sciences,  with  an 
admixture  of  a  few  individuals  from  engineering  and  a  few  from 
the  collateral  social  sciences.  Consequently,  reading  compre¬ 
hension  should  not  be  a  limiting  factor,  nor  should  vocabulary 
control  be  a  significant  problem. 

The  Guidebook  should  have  a  good  level  of  reader  "pull" 
because  of  its  inherent  high  degree  of  vocational  relevance. 

This  does  not  imply  that  stylistic  standards  can  be  relaxed,  but 
it  does  mean  that  the  format  can  be  unspectactular,  and  consequently 
more  economical  with  respect  to  cost  of  production. 

The  style  level,  in  fact,  should  probably  be  at  the  college 
textbook  level.  A  good  model  would  be  Scientific  American  or  . 
the  Bulletin  of  the  Atomic  Scientist  from  the  field  of  commercial 
periodicals  (as  opposed  to  professional  or  scholarly  journals) . 

The  physical  form  might  be  that  of  a  ring  bound  (i.e., 
looseleaf)  technical  report  because  there  is  a  strong  possibility 
that  the  Guidebook  will  need  to  be  updated  or  augmented,  or  both, 
over  the  span  of  its  intended  use- life. 

3.  Topical  Outline 

(Note  1:  Front  matter  is  described  in  "3.  Pagination") 

(Note  2:  The  book  as  a  whole  can  be  divided  into  three  major 
sections.  Each  section  will  contain  several  chapters,  but  each 
chapter  should  be  relatively  concise,  averaging  about  4-5  pages 
in  length,  with  no  chapter  longer  than  about  10  pages.  The  chapters 
would  be  "replaceable  units"  for  updating  purposes.) 


Section  I :  Orientation 


Chapter  1 .  Overview 

e  purpose 
e  substance 
e  layout 

Chapter  2.  Steps  in  the  standard  military 
system  acquisition  process 
e  mission  analysis 
e  concept  development 
e  configuration 
•  e  test  and  revision 

e  production  and  delivery 

Chapter  3.  Exceptions  and  variations  in  the 
system  acquisition  process 
e  administrative 
e  budgetary 

e  branch-of- service  linked 
e  technology  driven 
e  situational/idiosyncratic 

Chapter  4.  The  charter  documents  for  human  factors 

participation  in  military  system  development 
e  directives 
e  NIL-SPECS 
e  policy  papers 
e  other  (occasional)  documents 

Chapter  5.  General  roles  and  functions  involved  in 
the  human  factors  contribution 
•  technology  base 
e  planning 

* 

e  user  representation 
e  program  justification 


Chapter  6.  Variations  related  to  type-of -system 
e  vehicular 
e  ordnance 
e  C2  &  C3I 
•  etc. 

Section  II:  Working  Methods 


Chapter  7.  Specific  contributions;  human  factors 
principal  products 

e 


e 

e 


Chapter  8.  Pitfalls 


Chapter  9.  General  expectations  of  team  leaders 
e  combat  developers 
e  material  developers 
e  prime  contractor  -  project  directors 
e  subcontractors 

Chapter  10.  Specific  expectations  of  team  leaders 

• 

e 

e 


Chapter  11.  Integration:  roles ,  functions,  and  products 


Assessment  Methods 


Chapter  12. 


Chapter  13. 


Chapter  14. 


Evaluation  operations 
e  general-subjective 
e  technical-objective 
•  principles  of  benefit-cost  analysis 

Team  leader-initiated  evaluations 
e  instigation:  projective  and  retrospective 
e  procedure 

e  interpretation  of  findings 
Self-initiated  evaluations 


Chapter  16.  A1 


Chapter  15.  Impact  analysis  methodology 


ternative  methodologies 


Section  IV:  Summary  of  action  steos 


Chapter  17.  Getting  on  board 
Chapter  18.  Doing  the  job 
Chapter  19.  Proving  worth 


4.  Pagination 
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stock.  No  illustration; 
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unnumbered  1ft. -hand  page  -  Blank 

p.  7-1  (rt.  page)  -  Chapter  7  title  and  text 

(Note:  Chapters  follow  in  sequence,  with  each  chapter  beginning 
on  a  right-hand  page  and  pages  numbered  sequentially  within 
each  chapter. ) 

(Note:  Sections  follow  in  sequence,  with  each  section  demarcated 
by  a  colored,  unnumbered  overleaf  page  bearing  the  section 
number  and  title  on  its  front,  or  right-hand,  side) 

(Note:  Following  Section  IV,  Chapter  19,  an  index  to  the  Guide¬ 
book  is  provided.  This  section  should  also  have  an  overleaf  page 
on  colored  stock?  the  index  itself  will  be  two-column,  running 
2-4  pages. ) 


Back  cover 


Blank  on  both  sides 


APPENDIX  C 

PRINCIPAL  PRODUCTS:  ASSUMPTIONS  AND  ACTIONS 
Content  of  the  Role-of-Man  Statement 

A  statement  of  the  role  of  man  as  part  of  the  Mission 
Element  Needs  Statement  (MENS)  should  include  the  following 
considerations : 

Assumptions : 

e  A  separate  "role  of  man"  analysis  will  be  provided  for 
each  alternative  system  concept  selected. 

e  Human  engineers  will  develop  "role  of  mam"  concepts  and 
interact  with  mission  analysis  team  in  development  of 
MENS. 

e  "Role  of  mam"  components  aure  listed  according  to  probable 
order  of  presentation  in  MENS  (not  according  to  their 
development  sequence) . 

Actions: 

1.  List  effects  envisioned  for  overall  system  as  a  result 
of  role  of  mam  devised  for  each  alternative  system 
concept  as  configured  (e.g.,  operability,  maintainability, 
mission  effectiveness) . 

2.  List  effects  envisioned  for  man's  role/personnel  subsystem 
as  a  consequence  of  each  alternative  system  concept  as 
proposed  (e.g.,  safety,  habita&ility,  user  acceptance). 

3.  Determine  location  of  mam  in  system  to  perform  designated 
role. 

4.  Specify  advantages  accorded  mam's  role  for  each  alternative 
concept  (e.g.,  facilitate  operation  of  system,  allow 

for  contingencies) . 


C-l 


5.  Specify  disadvantages  accorded  man's  role  for  each 
alternative  concept  (e .g. ,  manpower  reserves  consumption, 
level  of  training  requirements) . 

6.  Determine  required  human  performance,  behaviors,  capabilities, 
and  performance  limits  (e.g.,  sensing,  processing,  information 
storage,  decision  making,  responding)  identified  for  each 
functional  category. 

7.  Determine  personnel  constraints  impacting  man's  role  for 
each  alternative  system  concept  such  as  the  following: 

a.  maximum  and  minimum  numbers  of  personnel  who  can  be 
used  in  the  system 

b.  types  of  personnel  (e.g.,  skill  level  and  aptitude) 
available  for  system  assignment 

c.  anthropometry  of  identified  personnel  population 
(existing  and  projected) 

d.  user  acceptance  problems  projected  and  their  effects 

e.  effects  of  system  and  mission  as  configured  on 
personnel  vulnerability  (e.g.,  environmental  hazards) 

f.  communication  requirements  and  limits  (system  and 
other  personnel) . 

8.  Determine  implications  envisioned  for  each  alternative 
system  concept  upon  requirements  for: 

a.  training  (e.g.,  level  of  training,  trainability, 
training  support  and  facilities,  training  devices) 

b.  manpower  (e.g.,  manpower  levels,  performance  availability) 

c.  life  support 

d.  "-ilities"  support  (e.g.,  logistics,  reliability, 
maintainability) 


e.  social/organizational  impact  (e.g.,  MX  basing). 

9.  Select  contributions  to  function  analysis  in  Mission  Analysis 
Phase : 

a.  identification  of  threat 

b.  need  demonstration:  new  system  or  modification  to 
current  system 

c.  requirement  . 

d.  mission 

e.  system  objective  definition  (and  required  input/output) 

f.  mission  segment 

g.  scenario (s) 

h.  functional  categories 

i.  functional  flow  and  operational  event  sequences 

j.  system  specification: 

1.  manual 

2.  hardwired 

3.  automated:  Facilitate  system  functioning 

Override  (bypass)  system  malfunctioning 
Control  system  graceful  degradation 
Permit  system  to  operate. 

10.  List  human  factors  characteristics  that  will  facilitate 
successful  system  development  and  mission  success  for  each 
alternative  concept  (design,  development,  testing,  production, 
deployment,  and  operation) : 

a.  advancement  in  state-of-the-art  human  factors 
technology 

b.  currently  available  human  factors  technology. 


11.  List  impacts  upon  cost  and  system  effectiveness  for  each 
alternative  concept  in  association  with  human  factors  inputs 

a.  R&D,  training,  personnel,  manpower 

b.  mission  success,  vulnerability,  survivability. 

12.  Prepare  human  Factors  R&D  Program  Plan  tailored  to  each 
alternative  concept  for  balance  of  system  life  cycle. 

Content  of  the  Allocation-of -Functions- to-Man  Statement 
■  as  Part  of  the  Decision  Coordinating  Paper 

A  statement  of  the  allocation  of  functions  to  man  as  part  of 
DCP  should  include  the  following  considerations: 

options: 

e  The  following  items  will  provide  direct  input  to  the 
specification  of  the  function  allocation  process: 

-  Mission  Element  Needs  Statement  (MENS) 

•  mission  scenarios 

-  functional  flow  block  diagrams 

-  mission  time  lines. 

e  Function  allocation  will  provide  support  to  the  proposed 
system  by  illuminating  the  following  criteria: 

•  system  performance 

-  cost-effectiveness. 

:  Both  criteria  have  as  a  function  human  performance.  Human 

* 

performance  can  be  specified  according  to  degree  of  detail 
available  about  the  system  mission  and  environmental  factors 


•  Function  allocation  will  detail  functions  involving 
both  operators  and  maintainers . 

•  The  following  general  process  is  assumed  for  the  function 
allocation  process: 

-  identify  and  allocate  tasks  and  functions  to  be 
assigned  to  all  personnel 

-  identify  required  equipment 

-  evaluate  selected  man-machine  combinations 

-  arrange  tasks  and  functions  to  maximize  mission 
effectiveness  and  reliability. 


Actions  z 

e  This  section  is  arranged  according  to  a  topical  development 
sequence  for  function  allocation  (not  development  sequence) . 

1.  Specify  human  factors  criteria  selected  for  allocation 
of  functions  (e.g.,  response  time,  error  rate  or 
human  performance  reliability,  cost) . 

2.  Specify  other  criteria  selected  for  allocation  of 
functions  (e.g.,  cost,  personnel  cost,  required 
training,  weight,  development  time,  development 
risk,  safety,  maintainability,  system  effectiveness, 
physical  volume  and  size  limits,  and  survivability) . 

3.  List  allocation  of  each  function  to: 

a.  one  or  more  opera tor s/maintainers 

b.  machine  only  (includes  automation) 

c.  combination  of  man  and  machine 

d.  function  currently  not  amenable  to  mam  or  machine 
performance. 
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4.  Multiple  operator/naintainer  and  man/machine  functions 
will  include  specification  of  the  type  of  redundancy  in 
the  task  being  proposed  (e.g.,  parallel  or  sequential 
mode ,  or  hybrid  of  both) . 

5.  Provide  estimate  of  feasibility  of  performance  for  each 
function  allocated.  List  the  effect  of  different  allo¬ 
cation  versions  upon  mission  success  (e.g.,  probability). 
Provide  estimate  of  workload  upon  operators/maintainers 

as  a  result  of  each  allocation  version  (at  least  nominally) . 
(At  this  level  of  development,  workload  implies  task 
difficulty  and  will  include  requirements  for:  precision, 
concentration,  criticality,  mission  priority,  and  task 
continuity  for  operators/maintainers  involved  in  each 
manned  function.)  Account  for  effects  of  user  acceptance 
for  each  allocation  version. 

6.  List  human  performance  capabilities  required  of  operators/ 
maintainers  for  each  function  involving  man  and  verify 
whether  or  not  man  can  perform  each  in  terms  of  required 
physical  and  mental  parameters  over  the  required  time 
period  and  within  the  anticipated  environment. 

7.  Prepare  rank  orders  for  candidate  allocation  combinations 
according  to  criticality  of  functions.  (Criteria  for 
criticality  will  also  be  specified.) 

8.  List  all  bottlenecks,  data  overloads,  acceptance  problems, 
and  other  mission-critical  faults  that  occur  as  a  consequence 
of  each  allocation  version.  Specify  the  means  by  which  each 
allocation  version  will  relieve  them  and/or  how  to  modify 
the  allocation  version  to  accommodate  them. 

9.  Prepare  a  comparison  matrix  which  exhibits  all  allocation 
versions  versus  the  selection  criteria  (entries  in  the 
matrix  are  estimates  of  absolute  performance  or  rank  for 
each  allocation  version  or  each  criterion  measure) . 


10.  List  preferred  manned  functions  as  well  as  other 
combinations  or  allocated  versions. 

11.  Provide  a  rationale  for  the  preferred  approach  and 
selection  to  justify  the  allocation. 

Content  of  the  Task  Analysis  and  Human 
Engineering  Requirements  Product 

A  documented  task  analysis  and  statement  of  the  system 
human  engineering  requirements  shall  include  the  following 
considerations : 

Assumptions : 

The  following  items  will  serve  as  input  to  the  process  of 
detexmining  human  performance  and  human  factors  engineering 
requirements : 

e  MENS 

#  DCP 

•  Products  of  function  allocation. 

Task  analytic  techniques  will  be  utilized  to  encompass  pertinent 
aspects  of  operations  and  maintenance  for  a  proposed  system. 
Requirements  for  human  factors  engineering  will  also  encompass 
operations  and  maintenance. 

Actions : 

1.  The  principal  product  of  the  human  task  analysis  portion 
of  this  phase  will  be  a  completed  task  analytic  package 
(including  static  and  dynamic  aspects  for  all  tasks) . 
Overall,  the  package  will  provide  the  following  data: 

a.  tasks  and  task  sequences  required  of  operators  and 
maintainers 

b.  actual  equipment  employed 


C-7 


d.  maintenance 


Techniques  utilized  to  derive  these  data  will  include 
procedures  such  as  the  following;  Behavioral  Task 
Analysis,  Operability/Maintainability  Analysis,  Hazard 
Analysis,  Workload  Analysis,  Task-Equipment  Analysis, 
Operational  Sequence  Diagrams,  and  Link  Analysis. 

The  overall  task  analysis,  including  task  descriptions, 
will  be  presented  in  the  form  of  flow  diagrams,  tabular 
presentations,  and  narratives. 

The  human  task  analysis  will  commence  with  a  summary  of 
gross  tasks.  This  summary  will  demonstrate  the  feasibility 
of  achieving  system  performance  requirements  as  well  as 
ensuring  that  human  performance  requirements  do  not 
exceed  capabilities.  In  addition,  the  effects  upon  the 
following  items  will  be  described; 

a.  manning  level 

b.  equipment  procedures 

c.  requisite  skills  and  training 

d.  communication  requirements  (between  operators  and 
operators  and  the  system) 

e.  logistics  support . 

The  human  task  analysis  will  specify  tasks  critical  to 
system  performance  as  well  as  evidence  to  support  its 
criticality.  These  tasks  will  include  but  ndt  be 
limited  to  the  following  data: 

a.  information  requirements  by  opera tors/maintainers 
(including  cues  for  task  initiation) 

b.  information  available  to  operators/maintainers 
c'.  evaluation  process 


d.  decisions  reached  after  evaluation 

e .  action  taken 

f.  body  movement  required  by  action  taken 

g.  workspace  envelope  required  by  action  taken 

h.  workspace  available 

i.  location  and  condition  of  work  environment 

j.  frequency  and  tolerance  of  action 

k.  time  base 

l.  feedback,  informing  operator s/maintainers  of  the 
adequacy  of  action  taken 

m.  tools  and  equipment  required 

n.  number  of  personnel,  specialties,  and  experience 

o.  job  aids  or  references 

p.  communication  required  (including  type) 

q.  hazards 

r.  interaction  of  multiple  personnel 

s.  operational  limits  of  personnel  (performance) 

t.  operational  limits  of  machine  and  software. 

5.  The  human  task  analysis  package  will  provide  the  results 
of  an  operability/maintainability  workload  analysis 
(including  the  interaction  of  multiple  personnel) .  The 
operability  analysis  will  detail  the  following: 

a.  design  goal-quality  of  information  throughput 

b.  predict  expected  quantity  and  quality  of  throughput 
operators  should  expect 

* 

c.  comparison  of  predicted  with  desired  throughput  and 
resolution  of  differences . 


The  maintainability  analysis  will  detail  the  following: 

a*,  design  goal — including  the  effects  of  automated 
maintence 

b.  predict  performance  times  for  correction  (including 
identification,  fault  isolation,  and  correction)  of 
system  malfunctions 

c.  compare  predicted  maintenance  with  goal  and  resolve 
differences. 

Develop  requirements  for  human  factors  engineering  by 
analysis  of  effects  of  critical  tasks  upon  system  and 
equipment  performance,  cost,  periods  of  peak  personnel 
workload,  conflict  situations  placing  demands  upon 
personnel  and  equipment  as  well  as  requirements  not 
previously  apparent.  In  addition,  life  support  charac¬ 
teristics  will  be  detailed  covering  but  not  limited  to 
the  following:  noise,  shock  and  vibration,  temperature 
extremes,  atmospheric  contamination,  toxicity,  electric 
shock,  mechanical  hazards,  electromagnetic  and  nuclear 
radiation,  explosion/fire,  pressure  and/or  decompression 

This  analysis  will  also  result  in  the  prediction  of  the 
probabilities  for  operator  and  maintainer  error. 

Details  to  be  included  in  the  error  analysis  are: 

a.  identification  of  the  locus  of  errors 

b.  malfunction 

c.  extreme  conditions  and  environments 

d.  effects  of  enemy  action 

e.  recommendations  for  avoidance  of  design-induced 
error 

f.  rating  of  error  likelihood 

g.  rating  of  error  criticality 


h.  estimate  of  seriousness  of  consequences  to  personnel 
and/or  equipment;  and  system,  subsystem,  and/or 
component  performance. 

7.  Additional  requirements  for  human  factors  engineering 
involved  with  development  of  procedural  documents, 
personnel  planning,  and  system  testing  will  be  developed. 
This  data  will  be  obtained  from  an  analysis  resulting 
from  the  compilation  of  task-related  data  into  preliminary 
operator/maintainer  procedurally  oriented  task  descriptions. 
(Especially  important  in  this  regard  would  be  the  deter¬ 
mination  of  system  and  personnel  performance  time  and 
accuracy  requirements  to  be  used  in  system  test  and 
evaluation.  A  sequential  analysis  of  the  operational 
sequence  diagram  would  provide  these  data  on  a  dynamic 
basis  suitable  for  this  use.) 

Content  of  the  Optimal 
Man-Machine  Interface  Design 

The  optimal  man-machine  interface  design  recommendations 
should  include  the  following  considerations: 

Assumptions : 

The  following  items  will  be  regarded  as  inputs  to  the  human 
factors  engineering  design  of  the  man-machine  interface: 

•  Design  criteria  documents  (e.g.,  MIL-STD-1472) 

•  Performance  specifications 

•  Drawings  and  data  (e.g.,  functional  flow  diagrams, 
schematic  block  diagrams,  interface  control  drawings, 
overall  layout  drawings) 


•  Human  factors  engineering  input  (e.g.,  task  analysis) 
converted  to  detail  equipment  design  features. 

The  following  processes  are  considered  characteristic  of 
this  phase  of  system  development: 

•  Human  factors  engineering  studies,  experiments,  and 
laboratory  tests  (to  resolve  human  factors  and  life 
support  issues) 

•  Mockups  and  models 

•  Dynamic  simulation  (necessary  for  detail  design  of 
equipment  requiring  critical  human  performance) 

•  Human  factors  engineering  contributions  to  detail  design 

•  Human  factors  engineering  contributions  to  manpower, 
personnel,  and  training  issues  as  a  consequence  of  detail 
design 

•  Human  factors  contributions  to  test  and  evaluation. 

Actions : 

1.  Effects  of  the  working  environment,  including  habitability 
and  operability,  will  be  presented.  These  effects  will 
cover  the  following  areas:  work  environment,  crew  stations 
and  facilities.  The  incorporation  of  human  factors  into 
the  detail  design  of  the  above  will  be  demonstrated  by 
presenting  detail  design  drawings,  specifications,  etc. 
for  the  following  three  conditions:  normal,  unusual, 
emergency. 

Topics  to  receive  coverage  will  include  at  least  the  following 

a.  atmospheric  conditions 

b.  weather  and  climate 

c.  range  of  accelerative  forces 


d.  acoustic  noise,  shock,  and  vibration 

e.  disorientation 
£.  accessibility 

g.  adequate  visual,  auditory,  and  physical  links 

h.  adequate  non-workspace  areas 

i.  psychophysical  stress 

j .  fatigue 

k.  clothing  and  personal  equipment 

l.  equipment  handling 

nu  chemical,  biological,  electrical,  electromagnetic, 
toxicological,  and  radiological  effects 

n.  illumination 

o.  sustenance,  storage,  and  refuse 

p.  safety  protection. 

2.  The  incorporation  of  human  factors  in  detail  design  of 
the  crewstation  layout/arrangement  and  of  equipment 
having  an  operator /maintainer  interface  will  be  demonstrated. 
This  will  include  the  presentation  of  drawings  illustrating 
the  inclusion  of  human  factors;  for  example:  panel  layout 
drawings,  communication  system  drawings,  overall  layout 
drawings,  and  control  drawings.  The  following  additional 
items  will  be  requisite  to  the  demonstration  of  the  inclusion 
of  human  factors  in  system  detail  design: 

a.  ingress  and  egress  to  workspace  and  facilities 

b.  a  list  of  panels,  racks,  controls,  displays,  and 
indicators  existing  at  the  time  of  documentation 
which  have  received  human  factors  approval 
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c.  rationale  of  human  factors  layou t/ arrangement ,  detail 
design  of  crew  station (s) ,  and  any  equipment  having 
an  operator/maintainer  interface 

d.  a  list  of  considerations  used  to  arrive  at  design 
decisions:  results  of  studies,  requirements  based 
on  task  analysis,  mock-up  tests,  mock-up  based 
decisions,  and  simulations 

e.  a  list  and  explanation  for  deviations  from  human 
factors  or  design  requirements  to  the  man-machine 
interface 

f.  sketches,  drawings,  and  photographs  of  required  or 
anticipated  panel  rack  arrangements  or  new  designs/ 
design  modifications 

g.  drawings  or  photographs  of  each  crewstation  design 
showing  locations  of  all  crewstation  panels  in  relation 
seat/operator  position. 

3.  The  inclusion  of  human  factors  in  design  considerations 
involving  the  interaction  of  maintenance  technicians  with 
their  respective  equipment  will  be  demonstrated.  In 
general,  this  will  depict  the  following  steps/stages: 

a.  recognition  of  malfunctions  (displays) 

b.  isolation  of  malfunctions  (troubleshooting) 

c.  fault  correction  (access,  removal,  and  replacement, 
repair) . 

A  human  factors  maintainability/accessibility  design  analysis 
will  be  presented  to  include  at  least  the  following: 

a.  preliminary  drawings,  sketches,  or  photographs  showing 
each  equipment  and  location  in  relation  to  surrounding 
equipment,  passageways,  and  structures  (this  includes 
ancillary  equipment  also) 
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b.  rationale  of  human  factors  design  of  each  item 
requiring  maintenance  as  well  as  presentation  of 
decisions  used  to  drive  the  decision  process  (e.g., 
MIL-STD-1472,  results  of  studies,  simulation,  mock- 
ups) 

c.  incorporation  of  maintenance  task  analysis 

d.  descriptions  to  include  but  not  be  limited  to  the 
following : 

e  physical  size,  purpose  of  support,  and  test 
equipment  required  for  maintenance 

e  maintenance  procedures 

e  relation  between  accessibility  and  failure  rate, 
service  frequency,  calibration  frequency,  and 
requirements  for  rapid  maintenance 

•  methods  used  to  determine  accessibility  for 
maintenance 

•  anticipated  maintenance  and  accessibility  problem 
areas. 

4.  Best  available  data  on  equipment  operating  procedures, 
operational  sequence  diagrams,  and  task  analysis  will  be 
provided  to  organizations  responsible  for  manpower 
development . 

5.  A  human  factors  test  and  evaluation  plan  will  be  prepared 
to  cover  the  following  general  concepts: 

a.  fulfillment  of  human  factors  requirements 

b.  conformance  to  human  factors  design  criteria 

c.  quantitative  measures  of  system  performance 

d.  detection  of  undesirable  design  or  procedural 
features. 
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APPENDIX  D 

IMPACT  ANALYSIS  STEPS 


Human  Factors  Impact  Assessment: 

Conceptual  Framework 

Basic  Framework  and  Steps 

Exhibit  D-l  outlines  the  basic  impact  assessment  framework. 
The  development  and  presentation  of  the  analysis  entails  ten 
steps  or  phases.  The  steps  are  presented  in  a  logical  sequence, 
in  three  groups;  but  in  any  one  analysis,  as  indicated  by  the 
dotted  lines,  it  may  be  necessary  to  repeat  several  steps  in 
different  sequences  to  refine  perceptions  and  assessments  of 
critical  issues.  Each  step  will  be  discussed  in  some  detail 
below. 

1,  Establishing  the  Problem,  Goals,  and  Criteria.  The 
objective  of  this  step  is  to  isolate  the  specific  issues  to 
be  analyzed,  to  bound  the  requirements,  to  specify  the  specific 
goals  and  objectives,  and  to  derive  the  decision  criteria.* 
Fisher  (1971),  Quade  (1975),  and  Goeller  (1976)  provide  useful, 
generic  guidance  for  this  step.  Specifically,  this  step  defines 
the  content  and  purpose  of  the  human  factors  product  to  be 
developed.  The  principal  human  factors  products  are  listed  in 
Chapter  2. 

This  step  is  one  that  should  be  recognized  as  a  variation 
on  the  generic  system  analysis  method.  The  rule  is:  look  at 
the  ends  first  and  work  back  from  those  ends.  In  human  factors 
terminology,  we  would  probably  prefer  the  sequence:  Goals, 
Objectives,  and  Outcome  Measures  (or,  for  the  latter.  Dependent 
Variables) .  However,  the  principle  of  going  from  the  broad  to 
the  narrow  and  the  idea  of  a  hierarchy  that  includes  more 


•In  this  discussion,  the  term  goal  represents  An  "end,*  objective 
a  "means”  (that  is,  a  specific  accomplishment  within  an  explicit 
time  or  cost  target) ,  and  criteria  represent  specific  decision 
conditions . 


"variables'*  as  that  move  is  made  is  a  common  link*  This 
arrangement  is  illustrated  in  Exhibit  D-2  in  a  particular 
cost-benefit  impact  assessment  convention  (Goeller,  1976; 
Ostrofsky,  1977). 

The  following  hypothetical  example  illustrates  the  use  of 
goals,  objectives,  and  weights.  Assume  that  the  system  under 
consideration  is  proposed  for  the  XYZ  main  battle  tank.  The 
major  goal  is  to  achieve  an  armored  fighting  unit  that  could 
defeat  its  hostile  counterpart  in  certain  tactical  scenarios. 

The  objective,  0^,  could  be  that  the  frontal  armor  would  hold 
against  80%  of  main  round  hits  (i.e.,  any  grazing  angle  greater 
than  ±5°).  The  objective,  02,  could  be  to  achieve  an  average 
first- round  time  advantage  of  3  seconds.  In  this  case,  02  could 
receive  an  a  priori  value  weight  somewhat  higher  than  O^. 

Criterion  C2^  (contributing  to  objective  02)  could  be  a 
maximum  turret  traverse  rate  of  »>20°  per  second.  Criterion  C22 
could  be  a  maximum  elevation/depression  rate  of  «>45°  a  second. 

In  this  case  the  criteria  might  be  assigned  equivalent  value 
weights. 

Several  attributes  of  the  hierarchical  setup  should  now 
be  clearer.  Specifically,  as  one  moves  down  the  structure, 
the  objective  measurability  improves.  But  more  importantly, 
the  actual  assumptions  about  performance  are  made  very  explicit. 
That  is,  the  design  assumption  clearly  is  that  if  a  given 
elevation/depression  rate  and  a  given  traverse  rate  are  achieved, 
a  given  first  round  time  advantage  will  result.  Not  only  is  that 
assumption  measurable  (e.g.,  by  computer  simulation),  but  the 
tentative  weight  assignment  is  also  similarly  measurable. 

Computer  simulation  would  permit  a  whole  range  of  permutations  on 
the  traverse  rates  and  elevation/depression  rates  to  be  explored. 


Exhibit  D-2 

Goal  Relevance  Tree  Hierarchy  of 
Goals  -  Objectives  -  Criteria 


0o  (WD)  MAJOR  GOAL 

r - 1 - 1 - 1 

0,1*,)  02(W2)  .  .  .  On(WM)  OBJECTIVES 

I - 1 - 1 

<91  •  •  •  C^IWg,)  CRITERIA 


RELATIVE  WEIGHTS: 

TOTAL  VALUE  -  Wq 
OBJECTIVE  WORTH  «  W, 
CRITERIA  WORTH  •  Wu 


and  a  very  close  approximation  of  the  relative  importance  of 
one  to  the  other  could  be  obtained.  Moreover,  the  weights 
could  be  revised  as  other  kinds  of  testing  were  done. 


A  notable  gap  in  the  above  synthetic  scenario  is  the 
lack  of  explicit  consideration  of  the  human  factors  aspects 
of  sighting  and  firing  the  main  armament.  For  example,  human 
factors  questions  would  arise  about  the  compatibility  of  a 
maximum  80°  traverse  rate  with  the  human  factors  requirement 
(hypothetical)  to  lock-on  to  a  target  on  the  first  traverse 
with  no  waver.  Human  factors  engineering  solutions  based  on 
traverse  deceleration  rate  damping,  sight  reticule  size,  etc., 
would  need  to  be  fitted  into  the  goal  and  objective-attainment 
relationship  as  constraints .  The  basic  message  here  is  that 
it  might  not  pay  to  have  a  relatively  high  traverse  rate,  if  it 
led  to  an  overswing  of  the  turret  9  times  out  of  10  because  the 
rate/velocity  dynamics  were  incompatible  with  normal  human 
(psychomotor)  tracking  capabilities. 

The  characteristics  of  the  appropriate  set  of  goals, 
objectives,  and  criteria  is  critical  to  the  effectiveness  of 
the  analysis.  Several  useful  discussions  on  this  process  are 
provided  by  Fisher  (1971),  Quade  (1975),  and  Ostrofsky  (1977). 
The  input-output  matrix. technique  used  by  Ostrofsky  (1977) 
appears  to  be  a  particularly  useful  way  to  structure  this  step. 
An  illustration  of  the  matrix  is  shown  in  Exhibit  0-3.  The  row 
headings  define  the  user  and  the  system  major  phases,  and  the 
column  headings  define  the  requirements  and  bounding  or  con¬ 
straining  conditions  (e.g. ,  resources).  Other  row  headings, 
such  as  those  employed  by  Ostrofsky,  couid  be  used  in  this 
same  format  to  formally  incorporate  human  factors  consider¬ 
ations  into  the  system  design  process. 
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Exhibit  D-3 

Input-Output  Matrix  for  Problem  Formulation 


Major  Systam 
Davalopmant  Phasas 

Inputs 

Outputs 

Intandad 

Environmantal 

Dasirad 

Undasirad 

Mission  Analysis 

Concapt  Davalopmant 

Damonstrations 
and  Validation 

Full-Scala  Davalopmant 

Production 

(Sourca:  Ostrofsky,  19771 
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The  output  from  this  step  is  a  problem  statement,  an 
input-output  matrix  for  bounding  the  design-analysis  problem, 
and  a  set  of  weighted  objectives  and  decision  criteria  to  be 
used.  The  problem  statement  is  an  issue  that  one  or  more 
human  factors  related  actions  can  help  to  resolve. 

2.  Defining  the  Alternative  Solutions.  The  objective 
of  this  step  is  to  generate  a  set  of  explicit  strategies  or 
alternative  solutions  to  resolve  the  problem  or  issue  identified 
in  Step  1.  For  example,  within  the  human  factors  principal 
R&D  product — development  of  the  role  of  man  as  a  part  of  the 
mission — alternative  crew  sizes,  mission  flexibility,  and  system 
recoverability  could  be  specific  considerations.  There  are  two 
major  ways  this  can  be  done.  The  first  is  to  specify  a  set  of 
alternative  design  configurations/characteristics  or  process 
changes  at  the  subsystem,  component,  or  function  level.  The 
second  is  to  specify  a  criterion  function  (see  Ostrofsky,  1977) 
that  incorporates  the  design  parameters  in  a  mathematical 
function,  and  to  exercise  the  function  to  determine  the  preferred 
design  or  system  specification.  Either  approach  can  be  used. 

The  former  is  more  common  and  straightforward.  The  latter  is 
typically  more  rigorous  and  requires  more  definitive  analysis. 

Making  the  decision  options  explicit  is  a  fundamental 
principle  of  systems  analysis.  He  can  illustrate  this  principle 
in  the  context  of  using  cost-benefit  impact  analysis  to  measure 
the  impact  of  human  factors. 

Methodologies  such  as  cost-benefit  analysis  are  being  used 
increasingly  to  support  system  design  decisions  and,  to  a  lesser 
degree,  to  support  the  management  decisions  in  system  development. 
The  application  illustrated  here  includes  both  types,  but  empha¬ 
sizes  the  latter.  Management  decisions  of  special  interest  are 
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those  concerning  when  particular  inputs  to  the  design  deliber¬ 
ations  should  be  encouraged,  and  how  much  investment  to  make  in 
each  potential  source  of  such  inputs. 

Por  illustrative  purposes,  then,  let  us  say  that  the  range 
of  options  available  to  the  Project  Manager  with  respect  to  when 
to  encourage  human  factors  inputs  is  given  an  initial  framework 
by  the  four  design  phases  previously  defined,  i.e.: 

e  Mission  Analysis  (MA) 

e  Concept  Development  (CD) 

e  System  Demonstration/Validation  (SD) 

e  Full-Scale  Development  (ED) . 

The  main  options,  then,  are: 

1 .  None 

2.  MA  only 

3 .  CD  only 

4 .  SD  only 

5.  ED  only 

6.  MA  and  CD 

7.  MA  and  SD 


16.  MA  and  CD  and  SD  and  ED  (all) . 

(In  the  higher-order  options,  the  question  of  relative  degree  of 
input  becomes  a  factor— but  that  factor  overlaps  with  the  allo¬ 
cation  issue  and  adds  a  complication  that  is  not  needed  for  this 
illustration.)  Thus,  in  this  illustration  there  are  16  distinct 
alternatives  for  when  human  factors  inputs  can  be  encouraged. 

It  is  sufficient  for  this  step  simply  to  enumerate  them. 


3.  Specifying  the  Baseline.  The  objective  of  this  step 
is  to  define  the  status  quo  conditions  relevant  to  the  analysis; 
namely,  the  baseline.  Projected  impacts  are  evaluated  in  terms 
relative  to  a  baseline.  For  each  system  development  phase,  a 
systems  baseline  must  be  defined.  Thus,  if  a  human  factors 
action  resulted  in  a  design  change  in  the  demonstration/validation 
phase,  the  baseline  for  the  succeeding  system  development  phase 
would  incorporate  that  change  because  it  had  already  been  accom¬ 
plished.  Thus,  the  baseline  is  generally  tied  to  a  phase  in  the 
development  cycle. 

The  baseline  provides  a  basis  for  the  projection  of  future 
conditions  in  which  the  human  factor  changes  under  consideration 
are  not  developed  and  implemented.  A  baseline  could  be  defined 
for  a  set  of  human  factor  impacts  when  the  individual  impacts 
cannot  be  isolated.  However,  it  must  always  be  defined  so  that 
the  impact  areas  and  metrics  under  consideration  are  explicitly 
identified. 

The  easiest  way  to  understand  this  step  is  to  make  the 
argument:  each  new  system  has  a  (more  or  less  direct)  precursor 
system  (or  systems).  The  baseline  rests  on  the  precursor  or 
composite  family  of  precursors  which  we  can  call  the  reference 
system.  In  most  instances,  the  reference  system  will  be  the  one 
that  would  be  used  to  perform  the  mission  if  the  new  system  were 
not  developed.  For  those  analyses  in  which  human  factors  are 
emphasized,  the  mission  compatibility  criterion  has  a  strong 
old-new  functional  similarity  aspect. 

The  following  discussion  illustrates  the  notion  of  the 
mission/functional  analysis  in  defining  the  baseline.  There 
are  two  analytic  substeps  in  establishing  the  baseline  for  system 
design  and  cost  projection  purposes:  Functional  Differences  and 
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Functional  Deficiencies.  The  first  entails  the  specification 
of  the  reference  system  similar  functions  and  any  technological 
differences  between  the  reference  system  and  the  proposed  new 
system.  For  example,  for  the  XYZ  tank,  the  reference  system 
would  be  the  operational  MYY  tank,  and  the  technology  differences 
that  impact  on  the  man-machine  functions  could  include  those  in 
the  main  armament,  armor  metallurgy,  turret  stabilization,  fire 
control,  and  propulsion  components.  The  functions  of  interest 
are  those  needed  to  operate  and  maintain  these  components.  The 
product  from  this  substep  is  a  reasonably  detailed  functional 
differentiation. 

The  second  substep  is  a  deficiency  analysis  of  the  reference 
system.  Again,  it  is  functional  deficiencies  that  count.  For 
example,  was/is  the  reference  system  deficient  in  maneuverability? 
In  what  specific  ways?  We  also  need  to  know  what  specific  human 
factors  related  deficiencies  were  brought  to  light  during  the 
field  use  of  the  reference  system.  Possible  source  data  for  this 
kind  of  deficiency  identification  could  include  the  complaints  of 
operators  and  maintenance  personnel.  Observations  of  the  actual 
behavior  of  crews  and  maintenance  units  in  action  could  be 
appropriate.  The  human  factors  specialist  could  go  through  dry 
runs  of  crucial  segments  of  operational  and/or  maintenance 
sequences.  The  product  from  this  substep  is  a  definitive  list 
of  deficiencies.  If  value  weights  could  be  assigned  to  each 
deficiency  in  an  unambiguous  manner,  this  could  also  be  useful. 

The  baseline  is  completed  as  a  step  in  the  overall  method¬ 
ology  when  the  array  of  technological  changes  and  reference 
system  deficiencies  are  put  together  in  such  a  way  as  to  give  a 
preliminary  picture  of  the  prospect  of  whether  the  technological 
changes  will  tend  to  ameliorate  or  accentuate  the  deficiencies  on 
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a  one-by-one  basis.  Thus  from  the  baseline  we  can  get  a  set  of 
assumptions  that  indicates  what  some  of  the  major  design  problems 
are  going  to  be  for  the  new  system  and,  importantly,  which  are 
likely  to  be  human  factors  related. 

4.  Preparing  the  System  Definition  Statement.  The  objec¬ 
tive  of  the  system  definition  statement  is  to  summarize  concisely 
all  the  essential  information  and  assumptions  about  the  subject 
system  that  are  necessary  to  conduct  the  impact  assessment. 

An  important  part  of  this  definition  is  a  historical  record  of 
the  evolution  of  the  system's  design  and  development,  and  the 
corresponding  impact  and  cost  estimates.  Though  it  will  not  be 
possible  in  many  instances  to  aggregate  cost-benefit/impacts  from 
system  development  stage  to  stage,  the  definition  statement  can 
provide  selective  evidence  of  the  role  and  contribution  of  human 
factors  R&D. 

At  a  minimum,  the  system  definition  statement  should  contain 
specifics  on  the  following: 

e  Mission  Profile  (What  is  the  system  for?) 

e  System  Performance  and  Operational  Characteristics  (What 
are  the  system  capabilities?) 

e  Acquisition  Program  Schedule  (How  is  the  system  to  be 
procured?) 

e  Deployment  (Peacetime  and  Wartime)  Plan  (How  will  the 
system  be  utilized?) 

e  Support  Concept  (Initial  and  Mature)  (How  will  the  system 
be  supported  and  maintained?) 

e  Logistics  Goals  (What  are  the  unique  logistics  related 
goals,  e.g.,  reliability?) 


D-ll 


•  Integrated  Logistics  and  Training  Considerations  (How  will 
the  operators  and  maintenance  personnel  be  trained?  How 
will  the  required  material  be  purchased,  managed,  etc.?) 

e  Human  Factors  Related  Issues  (What  operation  and  mainte¬ 
nance  considerations  can  affect  the  cost,  capability,  and 
compatibility  of  the  design?) 

The  first  seven  items  are  typically  called  for  under  current, 
recommended  major  weapon  system  acquisition  analysis  guidelines.* 
For  these  analyses,  we  have  augmented  those  guidelines  by  adding 
a  separate  discussion  of  human  factors  related  issues  that  should 
be  considered.  These  are  issues  that  would  be  noted  and  discussed 
in  the  human  factors  products  (e.g.,  role  of  man)  at  the  different 
system  development  phases.  The  outcome  of  that  consideration 
and/or  impact  assessment  should  be  reviewed  throughout  the  system 
development  stages. 

5.  Selecting  the  Impact  Areas,  Metrics,  and  Empirical 
Measures .  The  objective  of  this  step  is  to  define  the  system's 
life  eye1  cost,  capability,  and  compatibility  impacts,  metrics, 
and  empirical  measures  for  the  goals  and  criteria  identified  in 
Step  1.  Some  criteria  may  be  included  explicitly  as  cost  or 
empirical  metrics,  depending  on  their  specificity,  measurability, 
and  abstract  properties. 

Metrics  and  measures  used  to  define  the  specific  nature 
and  focus  of  the  human  factors  R&D  impact  must  be  tailored  to 
the  phase  of  system  development,  the  human  factors  product  form, 

*See,  for  example,  DOD  Directive  5000.1,  Major  System  Acquisition; 
5000.39,  Acquisition  and  Management  of  Integrated  Logistics 
Support  for  Systems  and  Equipment;  and  DOD  Instruction  5000.2, 
Major  System  Acquisition  Process. 


and  the  system-mission  characterization.  The  impact  area(s)  and 
associated  component  metrics  and  empirical  measures  comprise  the 
vocabulary  to  describe  the  effect  of  the  human  factors  related 
change (s) . 

The  three  generic  impact  areas — cost,  capability,  and 
compatibility — were  introduced  in  the  Phase  I  report.  In 

Exhibit  3-2,  each  of  the  impact  areas  is  shown  to  be  definable 
in  terms  of  a  number  of  metrics,  and  the  metrics  were  shown  to 
be  functions  of  combinations  of  empirical  measures.  The  generic 

hierarchical  relationship  also  is  illustrated  in  that  exhibit. 
Moreover,  the  measures  and  metrics  for  capability  and  compati¬ 
bility,  in  particular,  reflect  contemporary  usage  for  describing 
cause-effect  relationships  in  both  human  factors  R6D  and  system 
engineering.  In  general,  a  human  factors  related  change  that 
affects  capability  or  compatibility  will  also  affect  cost. 

The  set  of  vocabulary  terms  presented  in  Chapter  5  are  from 
our  preliminary  findings.  They  represent  an  initial  step  toward 
the  definition  of  a  formal  and  stable  set  of  terms  to  discuss, 
model,  and  communicate  the  effects  of  human  factors  related 
changes  in  military  systems  design  and  development.  Each  of 
the  impact  areas  and  their  component  metrics  and  measures  are 
discussed  briefly  below. 

e  Coat:  For  a  weapon  system  specific  setting,  the  cost 
impact  area  is  the  life  cycle  cost  of  the  system.  An 
example  of  cost  metrics  would  be  operations  and  support 
while  a  related  measure,  for  example,  would  be  Below 
Depot  Maintenance.  If  a  military  system,  other  than  a 
weapon  such  as  a  C^I  system,  was  the  subject  of  the  ana¬ 
lysis,  it  is  likely  that  some  different  cost  measures 
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would  be  required.  The  guiding  criterion  is:  select 
the  set  of  cost  metrics  and  measures  that  reflects  the 
significant,  relevant  costs  effected  by  the  human  factors 
related  changes. 

e  Capability :  For  a  weapon  system  specific  setting,  the 
capability  impact  area  is  the  mission  worth  of  the  system. 
A  preliminary,  empirically  derived  set  of  capability 
metrics  (e.g.,  availability,  reliability)  and  measures 
(e.g.,  mean- time- to-repair,  mean-time-to-failure)  were 
derived  during  Phase  I.  The  particular  combination  of 
measures  used  to  functionally  define  a  metric  is  dependent 
upon  the  system  or  process  being  analyzed,  and  the  various 
ways  the  effect  of  the  human  factors  changes  can  be 
measured . 

e  Compatibility:  For  a  weapon  system  specific  setting, 
the  compatibility  impact  area  is  the  physiological  and 
psychological  suitability  of  the  design.  A  background 
discussion  of  compatibility  metrics  (e.g.,  user  accep¬ 
tance,  motivation)  and  measures  (e.g.,  temperature, 
noise,  vibration  stress,  altitude)  is  given  in  Chapter  5. 
The  underlying  Motion  of  the  compatibility  impact  area 
is  that  many  human  factor  related  effects  are  not  easily 
assessed  using  the  same  quantitative  metrics  and  measures 
as  for  cost  or  capability.  For  example,  reducing  an 
operator's  stress  is  a  substantive  benefit,  even  though 
its  contribution  to  enhanced  system  performance  is  not 
directly  quantifiable. 

The  result  of  this  step  is  a  specific  set  of  vocabulary 
terms  to  be  used  for  describing  the  impacts,  and  in  selecting/ 
constructing  a  model  to  estimate  the  values  for  the  measures, 
metrics,  and  ultimately  their  effects  on  the  impact  areas. 


6.  Selecting/Constructlng  the  Impact  Assessment  Models. 

The  objective  of  this  step  is  to  derive  or  select  appropriate 
techniques  or  models  that  can  provide  both  quantitative  and 
qualitative  measures  of  the  cost,  capability,  and  compatibility 
impacts  expected  from  the  application  of  the  human  factors 
change. 

In  effect,  one  needs  to  relate  the  criteria  from  Step  1, 
the  information  from  Steps  2  to  4,  and  the  impacts  and  metrics 
from  Step  5.  Furthermore,  that  relationship  must  be  relevant 
to  human  factors  R&D  products  and  the  system  development  process. 
These  relationships  are  tailored  to  and  essentially  define  the 
content  of  the  human  factors  efforts  discussed  at  length  in 
Chapter  2 . 

A  reasonable  approach  is  to  utilize  Ostrof sky's  (1977) 
design  methodology  as  a  basic  procedure,  and  to  augment  it  with 
other  models  that  deal  explicitly  with  life  cycle  cost  and  system 
capability  measures.  (Examples  of  the  latter  are  Goclowski, 

1978;  Forster,  1974;  Fabbro  6  Fiorello,  1977;  AF-Logistics 
Support  Cost  Model,  Design-to-Cost  Model,  and  the  Mission  Success 
Completion  Probability  Model.)  In  addition,  there  are  several 
techniques,  other  than  Ostrofsky's,  for  evaluating  and  quanti¬ 
fying  (imposing  cardinal  measures)  on  essentially  qualitative, 
ordinal  measures.  Examples  are  Gardiner  (1979) ,  Saaty  (1979) , 
Quade  (1975),  Hays  (1975),  Dalky  (1969),  and  Linstone  (1975). 
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Briefly,  the  sequence  envisaged  is  as  follows  (Ostrof sky. 


a.  For  the  criteria  defined  in  Step  1,  specify  the 
underlying  parameters.  These  parameters  represent 

the  constituents  of  the  criteria  in  a  systems-component 
sense.  Each  parameter  is  classified  in  terms  of  being: 

-  measured  directly 

-  measured  from  a  model 

-  included  in  other  elements 

-  not  measurable  within  existing  resources. 

b.  Define  submodels  of  the  primitive,  measurable  elements 
to  define  functionally  the  higher- level  parameters. 

c.  Combine  the  submodels  into  an  overall  model  to  estimate 
each  criterion,  and,  in  turn,  an  aggregate  criteria 
function  for  the  overall  goal. 

While  each  of  these  steps  is  critical,  it  is  most  important 
to  understand  the  causal  linkage  between  the  elements,  which  can 
be  a  mixture  of  qualitative  and  quantitative  measures,  the 
parameter  submodels,  and,  in  turn,  the  criterion  function. 

For  a  "hard"  parameter  such  as  reliability,  the  linkage  between 
it  and  cost  and  availability  is  rather  well  understood,  and  many 
acceptable  models  exist.  For  the  "soft”  parameters  such  as  user 
acceptance,  the  linkage  is  not  nearly  so  clear.  What  is  required 
is  a  procedure  that  will  handle  both  quantitative  and  qualitative 
criteria  (and  their  parameters  and  elements)  in  a  systematic  and 
credible  manner. 

In  summary.  Step  6  puts  all  the  information  from  Steps  1 
to  5  into  a  formal  setting  with  functional,  causal  relationships. 
From  the  previous  steps,  we  have: 


(Step  1) 

•  A  set  of  goals,  objectives,  and  criteria  in  a  hierarchical 
array . 

(Step  2) 

e  A  listing  of  (management)  decision  options. 

(Step  3) 

e  A  specification  of  the  baseline  in  the  form  of  an  explicit 
comparison  between  the  reference  system  and  the  proposed 
new  system  with  respect  to  technological  differences  and 
functional  deficiencies  in  the  reference  system,  and  pro¬ 
jected  implications  of  such  deficiencies. 

(Step  4) 

e  An  overall  characterization  of  the  proposed  new  system 
and  how  it  is  to  be  operated  and  maintained. 

(Step  5) 

e  A  listing  of  critical  metrics  and  empirical  measures. 

The  model  used  to  put  these  elements  together  can  take  a 
number  of  different  forms,  depending  upon  the  system  development 
phase  and  problem  setting.  A  discussion  of  model  types  and 
selection  criteria  is  given  in  the  last  section  of  this  chapter. 

We  can  now  proceed  to  summarize  the  final  four  steps. 

7.  Collecting  and  Processing  the  Data.  Given  the  specifi¬ 
cation  of  the  impact  areas,  metrics,  and  the  model  form,  this 
•tep  provides  the  required  data  to  "drive"  the  model.  Frequently, 
the  lack  of  data  in  sufficient  quantity  or  detail  will  constrain 
the  nature  and  accuracy  of  the  cost-benefit  analysis. 


8.  Setting  the  Conventions  for  the  Analysis.  This  step 
specifies  the  conventions  or  ground  rules  used  in  arriving  at 
the  cost,  capability,  and  compatibility  impact  estimates. 
Conventions  for  cost  and  capability  analysis  should  cover: 

a.  Normative  projections 

b.  Constant  versus  adjusted  dollar  cost  estimates/ 
projections 

c.  Mature  versus  transient  system  characteristics 

d.  Personnel  budget  or  economic  costs 

e.  Capital  investment  leadtime  considerations 

f.  Relevant,  variable  versus  total  costs 

g.  Uncertainty  analysis  (including  technical  risk) 

h.  Presentation  and  documentation  standards. 

9.  Estimating  and  Evaluating  the  Cost  Benefits.  This  step 
provides  the  output  from  the  model  and  data  prepared  in  Steps  7 
and  8. 

10.  Presenting  and  Interpreting  the  Results.  This  step 
entails  preparing  the  presentation  (including  illustrations  and 
documentation  of  the  results) ,  identifying  the  requirements  for 
additional  analysis,  and  specifying  important  issues  that  have 
high  degrees  of  uncertainty.  An  important  part  of  the  presen¬ 
tation  is  a  description  and  quantitative  portrayal  of  how  the 
change  impacted  the  system  design  and  its  life  cycle  costs  and 
performance.  Where  feasible,  the  specific  contribution  of  the 
human  factors  change  should  be  isolated.  Often  it  may  not  be 
possible  to  isolate  the  impact.  In  those  instances,  it  may  only 
be  reasonable  to  make  the  comparisons  at  the  aggregate  or  systems 
level  (e.g.,  new  vs.  baseline),  and  to  infer  the  role  of  the 
human  factors  impact,  in  addition  to  the  standard  tabular  and 


APPENDIX  E 

PRELIMINARY  LIST  OF  METRICS 

System-Related  Terms  and  Associated  Dimensions  (Unit  of  Measure) 


ACCESSIBILITY 

ACCURACY 

CAPABILITY 

COMPATIBILITY 

CRITICALITY 

DURABILITY 


EASE  OF  USE 


FAILURE  RATE/FREQUENCY 


FIRING  RATE 
HABITABILITY 


MALFUNCTION.  SYSTEM 
INITIATED 

MEAN  FLIGHT  HOURS 
BETWEEN  MAINTENANCE 
ACTION 

MEAN-MAINTENANCE  TIME 


(MTBAMA)  MEAN  TIME 
BETWEEN  ANY 
MAINTENANCE  ACTION 


subjective:  satisfactory/unsatisfactory  use  of  ad¬ 
mission  to  various  areas  of  an  item 

probability  /frequency  of  documented  error 

subjective:  mission  objective  achievable  given  the 
condition  during  the  mission 

subjective:  ability  of  items  of  equipment  to 
coexist  (including  effects  of  temperature  and 
moisture 

subjective:  relative  degree  of  task  importance  for 
mission  success 

probability:  item  will  survive 

a)  its  projected  life 

b)  overhaul  point 

c)  rebuild  point 

without  a  durability  failure  (failure  that  causes 
an  item  to  be  rebuilt  or  replaced) 

subjective:  tasks  associated  with  simplicity,  reada¬ 
bility.  etc. 

1)  number  of  failed  items 

2)  number  of  affects  (out  of  tolerances)  per  month, 
weak,  hour,  etc. 

time  (measured  from  firing  to  reloading  of  weapon) 

subjective:  adaquacy/ease  of  space,  transport, 
watch  standing,  rest,  relaxation,  workspace 
and  access 

frequency  per  unit  time  (hours)  based  on  avail¬ 
able  reliability  data  &  maintenance  data 

mean  probable  flight  hours  between  maintenance 
actions 


1)  mean  hours  preventive  and  corrective  mainte¬ 
nance 

2)  tout  preventive  and  corrective  maintenance 
time  divided  by  total  number  of  preventive  and 
corrective  actions  during  a  specified  interval 

seme  at  MTBF  except  all  maintenance  actions  are 
collected  as  data 


(MTBF)  MEAN  TIME 
BETWEEN  FAILURE 


(MTBM)  MEAN  TIME 
BETWEEN  MAINTENANCE 

(MTBUMA)  MEAN  TIME 
BETWEEN  UNSCHEDULED 
MAINTENANCE  ACTION 

(MTTR)  MEAN  TIME  TO 
REPAIR 


<MTTRa)  MEAN  TIME  TO 
REPAIR  (ACTUALLY 
ACHIEVED) 


(MTTRp)  MEAN  TIME  TO 
REPAIR  (FLIGHTLINE) 


(MTTR,)  MEAN  TIME  TO 
REPAIR  (INHERENT) 


(MTTRq)  MEAN  TIME  TO 
REPAIR  (OPERATIONAL) 


(OPERATIONAL)  SUITABILITY 


(PILOT)  WORKLOAD 


PRODUCIBILITY 


READY  RATE,  OPERATIONAL 


1)  mean  time  a  system  functions  until  occurrence 
of  •  failure  requires  corrective  maintenance 
(characteristically  over  a  two-month  period) 

2)  total  functioning  life  of  a  population  of  items 
divided  by  the  total  number  of  failures  within 
the  population  during  a  measurements  cycle 
(time,  cycles,  miles,  events,  etc.) 

mean  of  the  distribution  of  time  intervals  between 
maintenance  actions 

same  as  above  except  only  unscheduled  mainte¬ 
nance  is  collected  as  data 


total  corrective  maintenance  time  divided  by  total 
number  of  corrective  maintenance  actions 
during  a  specified  interval 

total  corrective  and  preventive  maintenance 
time  divided  by  total  number  of  corrective 
and  preventive  maintenance  actions  during 
a  specified  interval 

mean  probable  time  spent  in  flightline  mainte¬ 
nance  before  system  is  returnev  *>  a  ready - 
for -operation  condition 

total  corrective  maintenance  time  divided  by 
total  number  of  corrective  maintenance  actions 
during  a  specified  interval 

total  corrective  maintenance  time  divided  by 
total  number  of  corrective,  preventive,  ad¬ 
ministrative,  and  support  maintenance  actions 
during  a  specified  interval 

subjective: 

1)  establishment  of  system  operability  in 
operational  environment  (within  stated 
constraints) 

2)  identification  of  adequate  instrumentation, 
comfort,  visibility,  handling,  etc.  of  systems 
by  personnel 

subjective:  degree  of  effort  required  to  accomplish 
a  specific  task 

(T&E  application):  subjective  ability  of  dif¬ 
ferences  between  prototype  and  production 
models  to  achieve  desirable  result  (as  a  result 
of  ECP  &  program  change  orders) 

%  of  assigned  items  capable  of  performing  an 
assigned  minion  or  function 


SAFETY 


SERVICEABILITY 

STANDARDIZATION/ 
COMMONALITY  OF  DESIGN 


1)  probability  of  injury  or  damage 

2)  subjective:  satisfactory/unsatisfactory  materials, 
fire  &  explosion  protection,  mechanical  & 
electrical  hazards) 

time:  ability  to  service  in  specified  interval 

degree  of  similarity  (lack  of  ambiguities)  of 
two  displays  designed  to  same  specifications 
and  standards 


SUBSYSTEM  EFFECTIVENESS  subjective:  the  technical  capability  of  a  sub¬ 
system  (RADAR.  FLIR,  etc...)  to  accomplish 
a  specific  task 

SURVIVABILITY  probability  that  a  system  will  withstand  hostile 

man-made  environment  and  retain  mission 
accomplishment  capability 

TIME,  DOWN  (DOWN  TIME)  time  (hours,  frequency,  duration)  which  an  item 

is  not  in  condition  to  perform  its  specified 
function 


TRANSPORTABILITY  subjective:  ease  of  transit,  packaging,  load/ 

unloading,  security  &  fastening 

WEAROUT  rate  of  increase  in  failure  rate  of  items  over  system 

life  (cycles,  time,  miles) 


E-3 


Personnel-Related  Terms  and  Associated  Dimensions  (Unit  of  Measure) 


ACCIDENT  RATE 
ACCURACY 


ANXIETY 

APTITUDE  AND  SKILL 

ATTRITION/TURNOVER 


number  per  specified  number  of  hours 

1)  kill/no  kill  ratio 

2)  %  correct 

3)  subjective:  associated  with  cognitive  skills 
(e.g.,  observing,  estimating,  detecting,  recog 
nizing,  positioning,  reading,  etc...) 

4)  measure  of  precision  and/or  timeliness  of 
performance 

subjective;  stress  factors  associated  with  pilots 
(e.g.,  training,  confidence) 

1)  testing  scores  (e.g.,  AFOT) 

2)  subjective:  low  vs.  high 

%  attrition -number  of  attrited  personnel  divided 
by  number  of  attrited  personnel  plus  number 
of  nonattrited  personnel 


DISSATISFACTIONS/ 

SATISFACTIONS 

EFFICIENCY 

ERROR  RATE  (ANALYSIS) 


ILLUMINATION  LEVEL 


subjective:  ratings  of  challenge,  personnel-job 
match,  perceived  degree  of  utilization 

rating  success  on  a  task 

1 )  mean  error  per  performance  time 

2)  percent  and/or  number  of  operator  error 
(e.g.,  forgetting,  accidents,  inability,  etc...) 

3)  analysis:  includes 

a)  amplitude 

b)  frequency 

c)  type 

d)  change  over  time 

1)  measure:  luminance 

2)  subjective:  number  of  lighting  deficiencies 


INJURY  subjective:  injury  type,  severity,  frequency 

MAINTENANCE  CORRECTIVE  number,  rate,  frequency  of  acts  performed  to 

restore  an  item  to  a  specified  condition 


MAINTENANCE,  PREVENTIVE  number,  rate,  frequency  of  actions  performed 

to  retain  an  item  in  a  specified  condition 


MALFUNCTION,  HUMAN  frequency  of  test  participant  (operator)  error 

INITIATED  resulting  in  system/item  malfunction 


(MOBA)  MILITARY 
OPERATIONS  IN 
BUILT-UP  AREAS 


1)  communications  distance  (limitations) 

2)  weapons  effectiveness 

3)  tactics  effectiveness 


MORALE 


MOTIVATION 

NIGHT  OPERATIONS 

NOISE/BLAST 

PERFORMANCE  TIME 
OR  RATE 

PRODUCTIVITY 

PROFICIENCY 

RADIATION 

REACTION  TIME 


STRENGTH 

STRESS.  GENERAL 

STRESS. TASK  OVERLOAD 

TASK  COMPLEXITY/ 
DIFFICULTY 

TASK  DURATION 
TASK  FREQUENCY 


subjective:  ratings  of  individual  parsonnal  identifi¬ 
cation  and  satisfaction  drith  work  group,  job 
activities,  duties,  supervision,  ate. 

subjective:  rating  of  desire  to  perform  duties, 
obtain  mow  ii  nr  a  f*wfw»a 

srirsun*  enpm  nviM|  uwvoinm 

performance  (target  identification)  in  night 
missions 

sound  pressure  measurements  (op.,  dbs.  amplitude, 
also  velocity,  wavelength  frequency  in  here) 

moan  time/number  per  tome  unit/rate 


units  produced  per  some  interval 
tost  scores  (written) 

radiation  affects  aircrew  performance  on  radiation 
environments 

1)  (time  reaction):  uptime  to  initiate  a  mission, 
measured  from  the  time  the  command  is 
received 

2)  operator  perception  time  (or  sun  time)  in 
response  to  some  initiating  stimulus 

amount  lifted  (kilograms) 

gas  (general  adaptation  syndrome) 

subjective:  workload  excessiveness 

subjective:  rating  based  on  knowledge  and  skill 
required  for  performance 

tuttl  time  required  for  task  completion  (also  as 
in  tracking  targets**  of  time  on  target) 

number  of  responses  made  by  an  operator(s) 
In  a  specified  interval 


TEMPERATURE 


TIME.  ADJUSTMENT/ 
CALIBRATION 

TIME,  CHECKOUT 


TIME,  FAULT  CORRECTION 

TIME,  FAULT  (ISOLATION) 
LOCATION 


measures  of  comfort  and  performance  in  variable 
temperatures 

time  required  to  make  needed  response 


time  required  to  verify  performance  of  an  Hem 
(in  specified  condition) 

time  required  to  correct  a  failure 

time  (hours)  measured  from  discovery  of  a  fault/ 
failure  to  correct  identification  of  faked  item 


TIME, TASK  TIME 
TIME,  TURNAROUND 


time  required  to  perform  task 


time  required  to  service  or  check  out  on  item 
for  recommitment 

USER  ACCEPTANCE  subjective:  underuse,  misuse,  abuse  of  equipment 

due  to  dissatisfaction  with: 

a)  machine  function 

b)  status 

c)  economic  fears 

d)  survival  fears 

e)  enjoyment  of  manual  performance  of  tasks 

VAPORS/EMISSIONS  measured  in  parts  per  million  (PPM)  over  specified 

time 

VIBRATION  frequency  (in  Hz)  over  a  unit  exposure  time 

WINDFORCE  (Q -FORCE)  wind  speed  indicator  (impact  on  physical  operating 

environment) 

subjective  level  of  effort  required  to  accomplish 
a  task 


WORKLOAD 


